Archive for the ‘Search’ Category

Siva Vaidhyanathan, author of the forthcoming book The Googlization of Everything, has posted on his blog what seems to be the entire text of one chapter, about Google in China. So it was frozen well before Google’s decision to stop censoring and perhaps abandon ship. It is a nuanced, balanced argument, with some compelling detail. He notes that censorship is not as simple as the “great firewall” metaphor would suggest, and that absolutist no-business-with-oppressive-regimes postures are not actually productive. Siva replays the debate in which he, I, and Esther Dyson participated, with an honest assessment of the two sides of the argument.

During that debate on National Public Radio in November 2008, Harvard computer science professor Harry Lewis accused Google of violating its “Don’t be Evil” motto by creating Google.cn along the very lines that the Chinese government demanded. “Their choice was, to accept the Chinese ultimatum or to go home. They could have gone home but they didn’t. They stated and built the engine as the Chinese wanted it.” Lewis concluded, “Google didn’t choose the lesser of two evils when faced with the Chinese ultimatum. It chose the more profitable of the two evils.” Now, Lewis was making a debater’s point because, well, this was a debate. The question before the two panels was not whether Google on balance does more bad than good or good than bad. It was whether Google lived up to its motto. The Chinese deal gives Google critics – and my debating team – an easy shot. Perhaps it’s a cheap shot. But that is what debating is all about.

Esther Dyson responded to Lewis. Dyson is known as one of the central visionaries of the information age. She has been present at the creation of many of the most important initiatives of the Internet, including the gestation of several search engines. She is one of the brightest and most influential thinkers about digital technologies and their effects on the world. Dyson understandably believes in the transformative, perhaps revolutionary, power of information technology. “The great virtue of the Internet is that it erodes power, it sucks power out of the center, and takes it to the periphery, it erodes the power of institutions over people, while giving to individuals the power to run their own lives. Google is part of that. It’s one of these things that shines light on everything, it enables people to find stuff out, it enables them to question what their governments are doing, and it’s absolutely wonderful,” Dyson told the debate crowd in New York City. “Google by its very presence and its operation, even if it’s incomplete, creates increasing expectations for transparency, it starts people answering questions. It gets them to expect to be able to find out stuff.”

As I wrote in Chapter 1, I was sitting at the opposite table to Dyson. I was on Harry Lewis’ side of this constructed event. If the question at hand was whether Google violated its motto, I have to come down on Lewis’ side, as I was in fact on Lewis’ side. But in the real world, debates like this don’t matter much. To the people of China, Google’s fidelity to its motto doesn’t make a bit of difference. In the real world, Dyson has a much stronger point. Google might raise expectations. Google might spark some young person in China to ask one more question about why she can’t read this or watch that. Some Google is probably a little better for China than no Google.

You can listen to the debate here. The front page includes a nice picture of Siva and me, ecstatic (and a bit surprised) at the moment the audience declared our team the winner.

So it is time for me to fess up. Siva’s description and assessment are accurate. In fact, when I was invited to participate in the event, I said I could argue either side. They wanted me on the pro side, which was fine with me—as Siva says, in the rhetoric of a debate, it’s the easier argument about which to wax oratorical. But the argument requires a great deal of subtlety, and Siva’s chapter gives the nuanced view.

He doesn’t say how he would revise it now that Google seems to have gotten fed up with Chinese shenanigans …

I wrote in Blown to Bits about Google’s decision to do business in China, in spite of the need for censorship at the direction of the Chinese regime, notwithstanding Google’s mission statement about making information universally accessible. I thought this was the wrong decision, and I have argued that position publicly (winning against Esther Dyson, among others).

Google has now announced that in light of massive cyber-attacks on its servers from China, including attacks aimed at compromising the Gmail accounts of Chinese dissidents, it is reconsidering its decision to censor, and is even asking whether it can continue doing business in China at all.

These attacks and the surveillance they have uncovered–combined with the attempts over the past year to further limit free speech on the web–have led us to conclude that we should review the feasibility of our business operations in China. We have decided we are no longer willing to continue censoring our results on Google.cn, and so over the next few weeks we will be discussing with the Chinese government the basis on which we could operate an unfiltered search engine within the law, if at all. We recognize that this may well mean having to shut down Google.cn, and potentially our offices in China.

This is a huge decision. While I disagreed with Google’s decision to cooperate with the Chinese censors, I respected the fact that they couldn’t lightly walk away from the biggest business opportunity in the world. I also respected the fact that even the limited search they were offering enlightened many people about things of which they would not otherwise be aware (except that they could always use Baidu, the Chinese search engine).

So Google gets lots of credit for their promise to monitor the information freedom conditions in China and for standing on the principle that even if cooperation with evil is sometimes justified in the interest of a larger good, there are limits, and China has breached those limits.

In the war for digital liberty, chalk this up as a battle won for freedom.

Google Alerts are emailed notices of new items on the Web relevant to your favorite search terms. I have alerts set on my own name and the names of my books, etc. It;s a way to keep up to date on, well, mostly on the careers of the other Harry Lewises — one is a police chief somewhere, one is a football player in some minor pro league, and one is a race horse.

You should set some up too, especially if you don’t get enough email. By setting up multiple alerts, you can keep your inbox quite full of stuff, some of which may even be relevant to your life.

I have the preferences set so I get alerts of blog entries mentioning me, and my own posts on this blog come back to me as alerts. Here is one I got this morning:

You will notice that the subject line, about a drug store, is not in the original post; nor is the phrase “buy viagra” which has been inserted into the text. I’ve checked the HTML code of the web page to make sure there isn’t any hidden text that Google picked up; there isn’t. There is no link to a drug store, either on the web page or in the alert. Click on the link in the emailed alert and you go to the blog, not to any drug store site.

Somehow, someone seems to have edited the alert, somewhere between where it was generated and where I received it. Can’t figure out why or how. If anyone has a bright idea, I’d love to hear it!

Adam Raff, a founder of Foundem, an Internet technology firm, makes the case in today’s New York Times for “Search Engine Neutrality,” which is kind of like network neutrality except that the nondiscrimination policy would apply to the way search engines return their results. As Raff states it, search neutrality means that “search engines should have no editorial policies other than that their results be comprehensive, impartial and based solely on relevance.” He objects, for example, to Google favoring its own map service over competing map services. And he objects to the way Google down-ranked his company’s product comparison service, which, he says, severely impacted its business.

Many of the points Raff makes are versions of thoughts in Chapter 4 of Blown to Bits, where we discuss the distorting lens phenomenon and an extreme case of search oblivion at the hands of Google’s ranking. (We also make the point, as Raff notes, that some of Google’s keyword auction technology was the invention not of Google but of Overture.)

But can search “impartiality” and “relevance” really be defined statutorily? I doubt it, or rather, I doubt we would want the hash that Congress or a regulatory bureaucracy would make of an attempt to regulate the semantics of the entire English language (and not just English). And lots of things affect Google’s rankings –see the Webmaster Help page, which includes advice such as not creating pages with little or no original content. I don’t think we want a legal entity judging whether pages were downranked for these or other reasons, or whether Google’s Safe Search filter has improperly omitted someone’s web page entirely.

In the presence of competition, none of this would be a worry. People would choose a search engine based on whether they liked the results it delivered, or perhaps on the basis of quality ratings by an organization such as Consumers Report. They could move if the search company changed their policy. The same is true with net neutrality, actually — the demand would not be so compelling if the number of choices of Internet services were not limited to one or two in so many places.

Monopolies are always dangerous, and this op-ed drives home that point. Not sure I am persuaded about the remedy, though.

Note: Any account written by an agent of a company unhappy about where its name turns up in Google searches should be regarded skeptically. There are lots of possible reasons for Google to downrank a site that have nothing to do with Google trying to gain an advantage in a new business sector, and Foundem’s web page design certainly doesn’t dazzle. Would love to know the full facts here, but I don’t.

Google’s search engine isn’t perfect because it can’t read minds. In a sidebar in Chapter 4 we note that a search for “spears” returns few results that aren’t about Britney or her little sister — anyone looking for weapons was pretty much out of luck when we tried it. Google tended to give the results that most people want most of the time, and that is far more likely to be Britney than a pointed pole.

In recent weeks — I first noted it a month or so ago — Google’s search results seem to be less monotonous and the results pages have started to include some phrases at the bottom pointing to less common interpretations of the search phrase. (See¬†Google tinkers with ’special sauce’ for searches.) So the top page of results for “spears” now leads with Spears Manufacturing (a maker of PVC piping) and includes a link to the Wikipedia page for the pointy kind of spears. And the links across the bottom of the page offer you searches for “spears weapons,” “greek spears,” and also “spears flash,” “spears underwear,” and “spears no underwear,” all apparently common searches for a particular subcategory of Britney material. Not sure if these links are intentionally to subsets rather than alternatives to the tyrrany-of-the-majority favorite.

Google, whose mission is to organize all the world’s information and make it universally accessible, has decided not to organize and make accessible the world’s scientific data. In the interests of economizing, it is canceling its scientific data service, which promised to store massive quantities of scientific date, from the Hubble telescope for example, for shared use.

Google offers lots of wonderful stuff “for free,” and it’s not surprising that in a recession the company is picking its shots. But as Wired reports, Amazon, which also offers cloud data services, is waiting in the wings and may rush in to fill the void.

Google prides itself on the objectivity of the algorithm it uses for ranking search results. No payment for placement, and no editorial judgments being made behind closed doors. A recent interview with a top Google executive is creating some buzz that this may be changing just a bit.

Google offers a feature that enables users to re-order their own search results — useful if you expect to search for the same thing again and want it to appear at the top of the list. You would also be able to indicate that you don’t want a particular result to appear at all in the future. These are the faint square boxes to the right of search results — an up-arrow with a horizontal bar over it to promote a particular result, and an X to eliminate a result.

These re-orderings affect only your own subsequent searches. At least that’s the way things work right now. Here is the crucial passage in the TechCrunch interview:

[Google‚Äôs Vice President of Search Product and User Experience ¬†Marissa]¬†Mayer also talked about Google‚Äôs use of user data created by actions on Wiki search to improve search results on Google in general. For now that data is not being used to change overall search results, she said. But in the future it‚Äôs likely Google will use the data to at least make obvious changes. An example is if ‚Äúthousands of people‚Äù were to knock a search result off a search page, they‚Äôd be likely to make a change.

Now that raises a couple of interesting possibilities, as reported by two industry critics. The first is that a new front will be opened in the cat-and-mouse game with the search engine optimization companies. Perhaps, for example, you can get your competitor knocked off the first page of search results by getting enough users to do so individually. Google’s engineers are smart enough to counter such simple tactics, but perhaps not slightly more ingenious measures.

The other possibility is that there are human beings reviewing the patterns of movements, and making editorial judgments about which should be incorporated into the general search results. Mayer does say “they‚Äôd be likely to make a change,” and while this is just an interview and she probably wasn’t choosing her words as though she was under oath, it’s an interesting question just how the decision to re-order search results in response to user actions would be implemented.

As we discuss in Blown to Bits, there have long been individual cases of editorial judgment, though most complainants about their placement seem simply to have lousy web sites by Google’s explicit standards. One has to wonder if this latest tweak isn’t going to open a major can of worms. Happily, it’s the sort of thing that can be tested quietly, and abandoned quietly if it doesn’t work out well.

Last week’s Intelligence Squared debate (about which I blogged earlier) is now up for viewing on YouTube, in 13 parts. You can find them by searching for “intelligence squared google”. My initial presentation is #4, and I respond to a question from Larry Lessig in #8.

Last night a team consisting of myself, Siva Vaidhyanathan (of UVa, author of Copyrights and Copyrwrongs and The Anarchist in the Library), and Randy Picker (of Chicago Law School) debated a team of Esther Dyson (author of Release 2.0), Jeff Jarvis (author of the forthcoming What Would Google Do?), and Jim Harper¬†(director of information policy studies at the Cato Institute). It was fun for everyone, I think. I could have argued either side, but I was recruited for the affirmative. I focused my argument strictly on Google cooperating with the Chinese government by producing a censored version of its search engine, which I rather too dramatically also referred to as an “instrument of thought control” and likened to a “brainwashing serum” that no responsible American pharma company would make for a foreign government. It was an Oxford-style debate; I took it as my job to sway the crowd and win the argument, without lying but perhaps by exaggerating if the other side would let me get away with it. I think several of the other participants took it rather more as an actual religious war.

In the pre-debate poll, the voting was very much against the motion; when the poll was repeated, it was a dead tie, 47%-47%, with 6% undecided. By the debate rules — winner whoever changes the most minds — our team won. Fitting, on the eve of the 40th anniversary of Harvard’s great comeback 29-29 win over Yale in football!

The debate is in the Intelligence Squared series. A bouquet to the sponsors and staff of the series; it’s a great thing to do. Last night’s will be up on Youtube by the end of the week and in an NPR one-hour edited version shortly thereafter.