Posted
by
kdawson
on Saturday January 13, 2007 @07:40PM
from the without-a-trace dept.

El Lobo writes with a sobering account of how Javalobby dropped off the face of Google last month. The site had been attacked by forum spammers and Google indexed some of their spew before the Javalobby guys could remove it. According to a post in Rich Skrenta's blog, Google is now the de-facto front page for the Internet, accounting for anywhere from 70% to 78% of the search market. The power this conveys is hard to overstate. From the Javalobby saga: "We had completely disappeared from Google's main index! If you run a website, then you know how serious a problem this is. On any given day over 10,000 visitors arrive at Javalobby as a result of Google searches, and suddenly they stopped coming! ... Suddenly we no longer existed in the eyes of Google."

Such cynicism; but you do have a low user ID, so I'll give it a pass as perhaps the voice of a soul beaten down by actual slashvertisements. Perhaps you should read the article and give the content a chance? Yes?

Well.........yeah. If you search for the exact term javalobby, there's a good chance that their website would come on top. More interesting would be some ambigious serach terms that would put it on top.

yeah, and if you search for KillerBob on Google, my site comes up at the front. If you type my real name, my personal website isn't even on the front page. On the second page, there's a couple of scripts I wrote over 10 years ago, and a story I submitted to BBSpot years ago, but my personal website still doesn't show up. Selection of keywords. If you type the name of any specific site, you'll get that site first. If you type what the site does, you may find that it's much lower on the page ranking. They probably aren't worried about traffic from people who search for the word "javalobby", because those people probably already know about their site.

They're worried about the people who search for terms like "java help", which is what somebody who *doesn't* already know about their site would be searching for. In my case, it's quite deliberate. I'm using robots.txt to tell GoogleBot to ignore my personal website. It's *personal*. All it is is an e-mail gateway, anyway; the blog is restricted access. There's no point in having it in Google, so the robots.txt reduces my daily traffic.

Just because they show up when you enter the name of the site doesn't mean they haven't lost lots of PageRank.

They probably mean that they used to show up when you searched for "Java", but because the spambots created so many outgoing links they lost their PageRank and now you have to search for "JavaLobby" to get them.

1. That a Java site not having as bad spam problems has likely gained notability to Google at the cost of this one.

2. That his site should be back in case he fixes his problems at the next Google spidering, at least if Google is consistent here, and I don't see why they shouldn't for the best of their search index.

Slashdot (and digg for that matter) only hurt the small personal and hobby sites run on the $19.99 hosted solutions. Traffic from slashdot to real sites running real businesses isn't all that much to write home about. Now a mention on Yahoo, that is serious traffic.

Most of the time I ended up on it when searching for hints on a particular problem. People just go to hang out on slashdot, but I imagine many people, like me, end up reading and posting on javalobby only in a context of a specific question.

Fair enough... if it has the answers, people will go to it no matter where google places it.

What gets me is how self-feeding all this infatuation with being on the first page of google has become. I'm not dissing google - I use it dozens of times a day at work - but the intrnet is much more than pagerank.

I agree. I don't see how anyone can complain about not being the top hit for some general term (even if it's java.sun.com for "java"). If you don't think the results are relevant - submit a more specific query. And I guess for general terms one can show a few hand-curated links on the top... and I think google already does something like that.

I can't say for sure, but maybe he's connected with some moron that phoned me the day after Christmas !!! about a domain I registered for someone else (obviously looked up my contact info via whois), and got pissed off when I told them that they don't need help gaming the search engines.

Is his phone number 425-882-8838?

BTW - if anyone else has been phone-spammed by these c*ck-gobblers at internetadvancement, please get in touch with me so we can file complaints about abuse of the information in the wh

I just typed in "Javalobby" in the Google search and their link came up on top.

If you know the site exists and what it's called, it's not very likely that you're going to be looking for it on Google. I think the idea is that Javalobby's copious articles had been showing up with good placement on Google, under more "generic" java-related searches (couldn't resist the pun). They were getting a great deal of traffic from these Google results because they'd worked very hard to build an original, content-rich si

If you don't know what the URL is exactly, you may pop the name into Google to find it first. That's very important if you're browsing at work and you don't want to pull up a web page of nude chicks serving java in a lobby.:)

Seriously, I know a lot of people who Google first to find the link to the website (i.e., type "CNN" to go to the CNN webstie). Some people are too lazy or ignorant to type out the full URL.

My joke site (SSLI: Search for Satanic Lyrics [dimspace.com]) used to be the number one result for "Satanic Lyrics, but about two months ago, ZAP! Gone from the frone page of Google. It's something like number 50 now, so instead of getting... ummm... three visitors a day, I get something like one a week:-)

I had a similar, but opposite experience. I started setting up Yet Another Job Site, but I never got around to making it useful (see Click. Hired! [clickhired.com]). Google decided that it sort of liked it for a while, sending some traffic my way. I went from making nothing on my google ads to a few bucks a day. It wasn't much money, but it was fun seeing the traffic come in. Then google decided it was the crappy site that it was and my traffic went back to its deserved trickle. I wrote an article about it with pretty graphs:

I'm actually experimenting with the google adsense myself this week on a new website. I tried to make a site based on what I wanted to see, and the info I wanted to be able to find. After week #1, I've had 250 hits and a total of 5 ad clicks. It's been a lot of fun while I've had some spare time, but I feel I may already have hit my max.Posting on slashdot gets me a few hits. Google refuses to show all of my site: only the front page, and the forums despite submitting a sitemap per the google specs.

I've got a dedicated server running a few over hundred domains. Some very well maintained and other not. The general consensus from the SEO bigwigs is that burst of traffic you'll see at first is from GoogleBot's spiders picking up your keywords. If you site hits on some good keywords with low PKI you'll see good traffic within a couple months. Once that traffic starts rolling in someone at Google may actually view your site. If you happen to get decent traffic from low PKI keywords you'll see your traffic

My joke site (SSLI: Search for Satanic Lyrics) used to be the number one result for "Satanic Lyrics, but about two months ago, ZAP! Gone from the frone page of Google. It's something like number 50 now, so instead of getting... ummm... three visitors a day, I get something like one a week:-)
I see similar traffic due to the fact that my site is the number 3 for PI to a certain number of decimal places.

I made a proposal in the W3C AC forum a week ago that would kill linkspam. So far I have not managed to

And it's already possible via trivial javascript to hide that part of the page from search engines, if you actually had a reason to do that....which harms accessibility... Not a bright idea really, since not only are you preventing some of your customers from seeing the site, but in many parts of the world this is actually illegal.

1. Move all forums to Javalobbyforums.com or equivalent
2. ???
3. Hire 'little people' in multicoloured pointy hats to help generate traffic for your site not that it is now google acceptable
4. Profit!

They have a script called "PudgeThis.pl" that takes every 3rd article and inserts a random error/non-factoid/dupe/non-sequiteur before it can be posted. Its the "magic sauce." It helps that it is written in perl - and that the original author was drunk as a skunk at the time.

Of course, on slow news days, they change the parameters from every 3rd article to twice per article... which expl

Maybe you should RTFA - they're not relying on Google for "advertising"Maybe you should RTFA - they DO actively monitor their forums. They deleted the messages very quickly - but too late, because Googlebot beat them to it.Maybe you should RTFA - they DO have a site that people care about and frequently visit. But they want people searching for solutions that appear in their FORUMS to find those postings via search engines.

either way it wouldn't be bad for google to have a monopoly. The bad thing is if google decideds to use their extreamley popular search engine to shut down competitors in other services. If google started a car company and shut down searches for all ford toyota chevy etc that would be considered an abuse of a monopoly or even a near monopoly.

If they could have implemented one layer of security or verification to prevent spambots from registering (similar to phpBB or vBulletin), they would have prevented all this. But they didn't. There is no image verification on their forum registration page. All it takes is a spammer with a source of disposable e-mails such as dodgeit.com to spam your page to hell.

If the forum isn't particularly time sensitive, how about just not serving recent forum posts ( 1 week) to the search engine spiders, which advertise themselves as being such, no?

I'm pretty sure that presenting pages to Google that are different from what regular people would see is already a breach of the terms of being listed by Google, and it's already resulted in sites being de-listed. (ie. If Google can't see what other people would see, how is it supposed to index and rank it appropriately for its

I bet you that with in two weeks after you install phpbb with captcha and email account verification you'll have spam bots/spamers registering and spaming your forums

Now THAT is BS. The only reason phpBB is penetrable is because their default captcha is EXTREMELY EASY to bypass.

If you develop your own proprietary and independent captcha (either with a stronger image verification system, or by requiring the user to answer an easy trivia question), you automatically prevent spambots from registering on

It's not really trivial to create your own custom spam security. Not everyone with a website has a programmer. Even if they did, the cost of combatting spam can be huge. On my little blog I created the entire thing from scratch in Rails using a novel form of spam prevention, I found one post where someone had gone through the effort to code up a spambot just for that one page (linked from reddit)!But the essence of your post seems to be that it's the website operator's responsibility, which I don't think

It's not really trivial to create your own custom spam security. Not everyone with a website has a programmer. Even if they did, the cost of combatting spam can be huge. On my little blog I created the entire thing from scratch in Rails using a novel form of spam prevention, I found one post where someone had gone through the effort to code up a spambot just for that one page (linked from reddit)!

You don't even have to code it yourself depending on what forum you're using. There are literally DOZENS of p

If they did, they'd be violating accessibility. Not only is authentication that the user is sighted against W3C's rules, but it's also a potential violation of disability discrimination bans.

This is worked around with an audio capitcha. Since users have access to two different choices for proving themselves as human, it is more than enough to cover most cases. In the rare event that a person is audially and visually impared, most likely there will be a helper that can answer the capitcha for them.

1) Google don't have a monopoly on search, unless you think 70-75% or so is a monopoly (I don't).

2) Search isn't everthing. Yahoo.com and msn.com were highly-popular websites, last I checked; people are free to advertise on them (and yes, I know appearing in Search results is free). Specifically for java-related issues (which is what TFA's website appears to be) there are various websites on which you can a

The problem is indeed deeper than just a headache for a webmaster or two. Let's face it: just as the desktop software market depends on MS Windows, and a lot of software companies will vanish overnight in case Microsoft introduced a new trick [like, signed - for a price - executables only, or backwards-incompatible API, etc], so the web now depends on Google. Should all the Google system administration team take a week off - and voila, you get no new customers, because they don't know where to go, and you're lucky if somebody from your old clients returns using his browser's history. Of course, there's Yahoo, MSN, Nigma, and a hundred of startups, but all of them combined hardly have the same significance that Google enjoys alone. So let's either keep our fingers crossed and hope that Google will not do anything more evil than it does now, or... heh, I don't really know even what else could we do.

From the title, I thought this was going to be finding a mirrored copy of your website after you stop maintaining it and your host drops you. But being nolonger indexed?? That doesn't make your site dissappear - what a drama queen. Untill Google becomes the only search engine, or becuase a government institution, people need to stop being so dramatic. Websites existed before search engines as far as I understand.

Even if Google was the only search engine, why does JavaLobby assume that they have a right to be near the top of the results? Their site had poor content on it and Google indexed them appropriately. It's the spammers that are at fault, not Google.

whilst some people may have a point about the *cough* slashvertisment this article has made me think about Google and monopolies, should I now change my search engine of choice because having many players in any market is better or is a monopoly acceptable when they are (pretty much) the best... even if they do sometimes change where, and if, they list sites

Maybe this is where Google needs to provide multiple indexing algorithms. The idea by giving different result types ( most linked, closeness to keywords, flashiness, highest rated, totally random, etc ), this would make it harder for site spammers to know which algorithm to be targeting.

I refuse to even click the link. This site, based on what I see here, deserves anything bad that happens to it. Millions of sites see their traffic rise and fall every day. And none of them take up our valuable time to post a sniveling bitch about it to the front page of Slashdot.

I don't care f'r Google for personal reasons undisclosed, so I don't use their products.

They're not MY de facto site, nor do I consider TFA any more than fanboy buzz. Just like other search engines we've used over the years of 'net usage, they're just the one on top right NOW. Give it 10 years. They might be the next big monopoly, or the next Webcrawler.

Personally, I prefer the meta-search engines; more baskets means more eggs.

In the comments are some strings that one writer of theirs expects to find on their site when searching google, but didn't. I just searched for the "jgoodies data binding" and their site comes up the 7th top level listing on the first results page.

It seems to me that google worked perfectly here. When 50,000 spam and phishing messages were posted to that site, the ranking of it went way down. When they cleaned them up, the site ranking came back.

What, would the site owners have google preserve their site ranking even though the content on the site went in the toilet? As a google user, I'm quite happy that google de-listed these folks for a bit, because otherwise these and other searches would have been severely polluted.

Try typing any mis-spelling of javalobby. Anything. Google offers you the alternative of 'javalobby'. They *so* do not recognise this website... so much so that they dare to *suggest* it as an alternative to a common mis-spelling of the forbidden site. Bastards! How deep does their vitriol run?

If you would have tried doing even a little research, you would have found out that Google penalizes hacked sites [mattcutts.com] and even makes an attempt to contact the webmaster to alert them to the problem. Not only that, they'll relist you if you remove the spam.

1. Fail to follow even basic internet precautions standard since 1998
2. Whine loudly on Slashdot when search engine behaves as advertised
3. Get lots of new traffic
4. Profit

This has occurred with Snowboarding2.com as well. It use to offer a subdomain feature where snowboarders could create their own website. A spammer used a few subdomains and had cialis and other drug links placed to it all over the net. The subdomain service was ended a year ago and all of those subdomains have timed out for over a year as a result yet the site continues to be sandboxed by Google. A site that was on the first page of Google results since '99 is no where to be found. There is a difference between showing up on page 10 and being sandboxed completely. You can type in snowboarding2.com itself into Google and the website itself does not even show up. Google has been contacted several times regarding this and nothing has been done. A link campaign was also performed to overpass the amount of bad links with good links and that search term to no avail. With the recent Google update it is now a PR0 website when it was a PR5 for a very long time.

I think that the article is overdramatic, and maybe a bit of self-promoting.

According to
ALexa [alexa.com]
(look at Reach), they dropped by roughly a factor of 2 to 3, from 100 to 150 per million, depending on the base period chosen, to
about 50 per million. A factor of three variation in site traffic over a few weeks is large, but it's not the end of the world.

"Google controls 50% of the world's searches. This famous website is so controversial that it has been banned by the most popular search engine in the world 'Google'. That's right. You cannot find alexchiu.com in Google system. Some very important people don't want you to know about Alex Chiu. Alex Chiu is on more than 30 TV interviews, 250 radio interviews, and in business ever since 1996. Yet AlexChiu.com cannot show up on Google?"

Good riddence! I dared criticise Java and OOP there and it started a long involved discussion. When the discussion ranked too popular on their traffic ranking system, the editors yanked it. They couldn't handle Java criticism so they pulled a "China".

Visitors from Google are not your real customers, they are more like guests. You should service them well, of course, and they do contribute a lot to your profitability (if you are commercial) or your popularity as a site. But your real customers are those who remember your URL by heart and visit you again and again, posting to your forum and buying from you repeatedly. You should focus on them. Have an pot-in mailing list where they can learn about your news and make sure they are interested to know wha

I have a website that got around 600 visitors a day with a certain domain name (.fr).When my host had to renew the subscription for the domain name, they didn't, even though I paid. Then someone "stole" the domain name when as it was free.Now I had to buy another domain name (.com) and some a**hole put ads on my former domain.

Does anyone know what I can do to have google index the new one and give it the position my former domain name had ?

This happened last year. There've been several follow ups to the original blogpost on how the situation was resolved. There's even a guy from google who showed up in the forum and offered his helped to fix things. By now the damage has been undone of course but for some time Google stopped returning any javalobby results.

Btw. it's javalobby.org, they're not for profit and sort of pay the bills with the advertisements. Just like slashdot. I've been a member of their site since 1998 and they're good guys.

No, but they did do their best to stop OEMs installing other browsers. And there were issues with API disclosure and "sharp" business practices. Microsoft were basically abusing their monopoly. Lowering the index ranking of a site because that site has temporarily been spammed is not abusing Google's near monopoly - especially as there is every chance that a cleaned up JavaLobby could rise back up the rankings.

Google doesn't engage in the illegal activities that MS does, such as discounts for only offering windows on machines. Also, google doesn't bundle their services or products with any desktop operating system.

I'm referring to the Nokia 770, which comes with Google Talk, a search applet with Google as the default search. Technically, even the above is incorrect, if you fire up Ubuntu for example, the default search in the default browser is Google right out of the box. However, Goo

You've confused "monopoly" with "being successful." There's a big difference between the two. "Monopoly" implies that the user effectively has no choice but to use your product. Microsoft had a monopoly in Windows because a lot of commercial software and a lot of drivers were available only for Windows, locking the user in to that OS. Google doesn't even have a monopoly, since there are alternative search engines that can be used just as easily.

You're misunderstanding who the user of Google is. Don't worry. Most slashdotters make this mistake.*You* are not the user of Google - You're the *product* sold by Google. The real users are the websites that are advertised by Google.

I don't know what % of the *on-line advertising market* Google controls, but if an anti-trust case were to be made (ie: advertisers have to play by Google's unfair rules in order to have an on-line presense), it'd be through that angle, not by allegedly controlling the "on-line

And guess where live.maps.com is on Google's search?
Go look... no it's not on the first page....
Go to the second page of results... Ah yes half way down.... HMMMM
I think Google has a case to answer here, I simply don't believe Microsoft maps can possibly legitimately be ranked where it is.

Because you are an idiot. Go back to live.com and see where it shows up in the *search* results for maps (sponsored links DO NOT count, duh!). I tried, and the site appears nowhere in top TOP 50 results.

Hilarious, come on all you Google fanboys/MS anti-fanboys.... try and spin this one into yet another Microsoft bashing session I dare you, then I can see something truly imaginative.

I think Google has a case to answer here, I simply don't believe Microsoft maps can possibly legitimately be ranked where it is.

Since this is the first I had heard of Microsoft's map site, I am not struggling with this so much. First, if you search for "map", it's at position six. Looking at the Alexa comparison [alexaholic.com], the Wayback Machine [archive.org] (compare with the one for Google Maps [archive.org]), and the Wikipedia history [wikipedia.org] for Microsoft's maps, this all seems appropriate to me.

And as the clincher, the SNL skit Lazy Sunday [wikipedia.org] mentions M

Actually, going a bit offtopic here, but whilest Google has the most userfriendly interface, the actual maps they present are pretty crap. MultiMap are a lot better, with scans of OS maps. Google's UK maps don't even have motorway junction numbers on them.:(

Uh... it's in the domain name. It's also in text on the page.
On that arguement how can you explain maps.live.com being listed in google's results for maps at all? It's obviously a key word for the page. Last time I read anything about google's ranking algorithm it uses links from other sites to determine page rankings not the title of the page. On that basis I find it hard to believe sites like www.lib.utexas.edu/maps, www.whereis.com.au/ and www.eduplace.com/ss/maps/ can be ranked or linked to more th