Tuesday, March 20, 2007

Why the Semantic Web Will Fail

Don't get too excited by the title. But I do want to share a few thoughts...

It was running through my head just now, the work that we were doing here in Moncton to work on an e-learning cluster. Because I saw that 'cluster building' is still one of the major pillars of NRC's strategy, and I was wondering whether our work would ever be a part of that again.

And I was thinking about some of the things that didn't go so well in our first few years. Some companies went under - a couple, before we even talked to them, another, after we were in a project with them. And then there was the company that we sat down with, oh in 2002 or 2003, and laid it all out - RSS, content syndication, social networks. The whole Web 2.0 thing.

And they weren't interested. And in less than a year, they were gone.

And I thought about where we're right today and where we might be wrong, and why. Because despite having a pretty good track record (check for yourself, it's all on the public record - this year's predictions (bucking everyone else) include OpenID and the runaway success of Wii).

And I'm saying the semantic web won't work. Can't work.

But how do you explain that intuition?

And I was thinking about the edgy things of Web 2.0, and where they're working, and more importantly, where they're beginning to show some cracks.

A few of key things today:

- Yahoo is forcing people to give up their Flickr identities and to join the mother ship, and

- MySpace is blocking all the widgets that aren't supported by some sort of business deal with MySpace

- the rumour that Google is turning off the search API

And that's when I realized:

The Semantic Web will never work because it depends on businesses working together, on them cooperating.

We are talking about the most conservative bunch of people in the world, people who believe in greed and cut-throat business ethics. People who would steal one another's property if it weren't nailed down. People like, well, Conrad Black and Rupert Murdoch.

And they're all going to play nice and create one seamless Semantic Web that will work between companies - competing entities choreographing their responses so they can work together to grant you a seamless experience?

Not a chance.

Now - there are many technical reasons why I think the Semantic Web is a loser, along with some cultural and philosophical reasons. Namely: the people who designed the Semantic Web never read their epistemology texts.

But the big problem is they believed everyone would work together:- would agree on web standards (hah!)- would adopt a common vocabulary (you don't say)- would reliably expose their APIs so anyone could use them (as if)

Shall I go on?

So...

Maybe we won't be building clusters in Moncton, maybe we will. I don't know - I'd like to keep trying. Maybe people will listen to us or maybe (more likely) they won't.

The future is not in the Semantic Web (or in Java, or in enterprise computing - all for the same reason). Careers based on that premise will founder. Because the people saying all the semantic-webbish things - speak the same language, standardize your work, orchestrate the services - are the people who will shut down the pipes, change the standards, and look out for their own interests (at the expense of yours).

I don't trust any of them. Not even as far as I could throw them. Because I know they'd sell me down the river in a minute, if it meant one iota of business advantage. You know this too.

Yeah - we'll play games on Yahoo, create a not-too-serious blog with Google, post some tunes on MySpace (under an alias of course), and mess around with some photos on Flickr.

And we'll even go along with some unimportant things, like the university account and email, so we can access the course notes on Blackboard. The personal email address, that we got from our ISP, we will tell only to our closest friends - and we'll use the gmail account for logons and the Yahoo identity for spam.

We'll post to these Web 2.0 sites, but if the content means anything, we'll keep a copy on our computer as well (until Windows crashes and eats all your data, that is).

But trust them? Not a chance.

The future of the web will be based on personal computing.

Not because everybody in the world is some sort of Ayn-Rand-clone backstabbing money-grubbing leech.

But because there's just enough of them - and they're the ones who tend to rise in business. And when they say "give me your data" (or "let me manage your money" or "base your career on my advice") it's merely a prelude to their attempting to take you to the cleaners.

If my online world depends on them - and in the Semantic Web, it would - then my online world will fail. Will be a house of cards that will eventually collapse.

Yeah - I know. It's not a technical argument. And it probably reveals some of my own biases. But I can't shake the intuition that I'm right here.

76 comments:

Rather than expecting a top-down implementation coming from the established players, if there is any great utility to be gained, I think it will emerge from the ground-up in an initially limited yet scalable form.

But you're wrong: TCP/IP, HTTP and HTML are examples of technologies which are pretty well standardized and work across the industry - relying on everyone playing by the same rules.

It's in everyone's interest that plumbing works. It's just that what is considered to be plumbing varies over time - ten years ago there was fierce competition on HTML standards, with every browser manufacturer adding their own things. Now there is very little, as the competition has moved to other things.

When (if?) Semantic Web (or any other technology - Web 2.0 is not the Semantic Web) becomes a key part in the plumbing required to run the money-making applications, it will stabilize and automagically standardize itself.

I agree with your general premise in terms of today's computing paradigms. But to sit back and say it will fail is defeatist.

I am old enough to recall the days of private BBS systems run by hobbyist/enthusiasts, and used by the same people. These folks were the early adopters of the "social" internet (usenet readers, MUDD players, ...). After the researchers did the science, these are the people that did the technology and created the killer apps of the day.

I am also old enough to remember when the Web first came around, corportations didn't see any purpose in it until a business model was developed that showed them how to profit in that environment.

If anything I think the W3C has chosen the only likely route to success - define that the infrastructure standard (but mutable), and leave the implementation otherwise wide open.

Given the current state of the science, I think we are years away from the realizing the benefits of the Semantic Web (I define it differently than Web 2.0). I think it is much too early on to commit infanticide on this line of work. Corporations are settling back on their market positions and are comfortable with their web marketing plans to make significant investment in Semantic Web.

When enough of the technical roadblocks are overcome to admit the enthusiast / hobbyist community to enter the stage, and the "real" Semantic Web moves out of research and into the domain of the average techie-geek, the real killer apps will not be too far behind.

The question is, which of the sleeping giants will awaken in time to take advantage of it. The main question I have is will I have the foresight to realize it as it is happening?

"But you're wrong: TCP/IP, HTTP and HTML are examples of technologies which are pretty well standardized..."

I'm not wrong.

Packet switching (which is what TCP-IP does) was vigorously resusted for many years (compare, for example, the standards used by, say, Novell Netware or Windows Networking, not to mention the systems used by telephone systems).

Were it not for the fact that it was the design for the government-backed national research network, it would never have taken hold.

HTTP is currently under threat from such things as handle: and info: which would privatize the basic transport protocols of the internet.

HTML never properly standardized - as any developer coding for Netscape and Internet Explorer will tell you.

Finally: let me observe that none of these things ever became standards, in the formal sense of the term. They are, at best, protocols - that is to say, you use them if it is to your advantage to use them, not otherwise.

I agree with Tom Briggs - that's why I said "The future of the web will be based on personal computing."

And it is illustrative to see what the hobbiests are doing.

And what they are doing is not the Semantic Web. It's everything but - it's RSS, OPML, Wikis, Microformats, JSON, you name it. It's Web 2.0 - which, agreed, is quite different from the Semantic Web.

I don't know of anyone defining a canonical vocabulary in their spare time - but I know of numerous corporate projects (just have a look at OASIS for all you want (OASIS, because W3C requires that you cannot sneak any IP into the mix, and the cut-throat corporates aren't willing to agree to that)).

maybe we'd have to look at a funny field of science called "economics" that deals with the questions like "what is the cost vs. advantage of collaboration between businesses?" or "what are the conditions to foster collaboration vs. competition?". It's the same guys (Harvard Business Review) that advise M$ not to extend on this "genuine advantage" in order not to loose users of their OS...

The biggest driver for collaboration currently is to tap into a large group of users that are using a certain standard. So, the MySpaces are basicly generating a community of corporates all talking all the same language.

The interesting fact about companies is that, they may look almighty and powerful but, are (by nature) absolutely dependent to their customers.

When an open source, scalable and expandable semantic web platform will be created, which will offer secure information transaction via encrypted data and digital identities (the Trust layer), then the companies will be forced to adjust to it (see OpenID).

The users already demand for a unified way of accessing all those communities and websites out there and the Semantic Web can provide this common ground. Maybe they'll call it "Semantc" [sic] or "OurWeb" to make it sound more appealing but, without ontologies, words are just combinations of letters; meaningless.

"But the big problem is they believed everyone would work together:- would agree on web standards (hah!)- would adopt a common vocabulary (you don't say) - would reliably expose their APIs so anyone could use them (as if)"

One of the features of the W3C's model (based on RDF) is that it doesn't push the idea that everyone should adopt the same vocabulary (or ontology) for a domain. Instead it offers a way to publish vocabularies with some semantics, including how terms in one vocabulary relate to terms in another. In addition, the framework makes it trivial to publish data in which you mix vocabularies, making statements about a person, for example, using terms drawn from FOAF, Dublin Core and others.

The RDF approach was designed with interoperability and extensibility in mind, unlike many other approaches. RDF is showing increasing adoption, showing up in products by Oracle, Adobe and Microsoft, for example.

If this approach doesn't continue to flourish and help realize the envisioned "web of data", and it might not after all, it will have left some key concepts, tested and explored, on the table for the next push. IMHO, the 'semantic web' vision -- a web of data for machines and their users-- is inevitable.

"ok, aside from being defeatist, will you now go and re-learn what SEMANTIC means? properly structured information can be used for whatever purpose as long as it is available."

I think the point was that it wasn't going to be available, not that the technology is never going to work.

"... and did you realise publishing this on a blog is a little ironic?"

semantic web != web 2.0. When you're trying to be clever, it helps to actually be smart.

I think malware and Wikipedia are really telling examples of why the semantic web won't work - it's too easy to muddy the data with deliberately bad information, especially if it's in people's financial interests to do so.

I'm interested to see the predictions that you mentioned about OpenID and the Wii. Do you have a link?

At the risk of sounding like a smartass, the term 'semantics' has many different meanings. Even without rigorously defined, commonly agreed upon ontologies, you can still have semantics. But let's start with natural language semantics. Once you make some progress there, I doubt you'll need all of the machinery associated with the "semantic web" (whatever the hell that is).

One of the features of the W3C's model (based on RDF) is that it doesn't push the idea that everyone should adopt the same vocabulary (or ontology) for a topic or domain. Instead it offers a way to publish vocabularies with some semantics, including how terms in one vocabulary relate to terms in another. In addition, the framework makes it trivial to publish data in which you mix vocabularies, making statements about a person, for example, using terms drawn from FOAF, Dublin Core and others.

The RDF approach was designed with interoperability and extensibility in mind, unlike many other approaches. RDF is showing increasing adoption, showing up in products by Oracle, Adobe and Microsoft, for example.

If this approach doesn't continue to flourish and help realize the envisioned "web of data", and it might not after all, it will have left some key concepts, tested and explored, on the table for the next push. IMHO, the 'semantic web' vision -- a web of data for machines and their users-- is inevitable.

Everything's going to happen in the long run. In the long run-- heck in the medium run-- we get to turn as much as we want of the local matter into computonium. (Computonium = the maximally efficient arrangement of matter in this universe for performing computation. Undoubtedly it would take less than a pound of computonium to render our civilization, much less this whole conversation, very irrelevant.)

So the question isn't whether we'll ever create a semantic web, the question is whether we'll do it soon enough to have historical importance. Seen that way, I almost agree with this blog post. There is something fishy about the project as currently envisioned & manifested, and it does have this sense of futility about it.

So on the one hand it's inevitable that we're going to have richly structured information-- sometime. On the other hand, the practical efforts in that direction are spinning their wheels in the mud. Isn't it obvious what to hope for? We've got a lot of smart people here together working on this. There is certainly more than one way to go about it.

I've had a nagging worry for some time about entrusting all my data to the same folks who provide me those nifty free online applications like Google Docs, and Zoho, and even My Yahoo (that venerable old goat).

In my (still rather vague) notion of "the way things should be" I envision using all those nice online tools, but having them use my browser as a conduit to connect with a back-end storage service that I control, a la running my own OpenID server to control my own identity.

Basically, I want to connect to Google or Microsoft or Joe Somebody's server and slurp down an application to run in my browser (in a framework like YouOS that will permit that application limited access to any others that might also be running in my browser [or elsewhere on the net] simultaneously), but once it arrives, I want it to use the datastore I will point it at and validate with on my own for data persistence.

And of course I will only use Google's, or Microsoft's, or Joe Somebody's applications if I trust them -- which probably means if they are open-source and vetted by a large user community, and if their hosts haven't tarnished their reputations at any point with business-driven shenanigans or gross security failures.

I think the biggest problem that the semantic web has is ... well ... semantics.

I would argue that there is no single version of the truth to build the semantic web upon. Or, there may be a single version of the truth, it is just that every person has a different view of it. Truth is inherently dependent on perception, which is strongly influenced by group dynamics. Any approach that doesn't consider this is doomed to obscurity.

Interesting points. It seems that there may be a fine line between competitive practices that leverage absolute control, the need to inter-operate to remain relevant and the peril of too open anarchy.

HTML is a good case in point. Microsoft abused its monopoly position by not updating its web browser and saw its market share start sliding away to Firefox. Microsoft had to get on board again with both features and standards compliance.

On the other hand Linux, which is arguably the most open of systems can't get enough momentum pointed in one direction because nobody can step forward and say this is the way it will work, at least on the consumer interface level.

But the real problem with the semantic web, at least how it is typically defined is that there just isn't enough incentive to make websites semantic or as you define it inter-operable. There is a need to maintain the delicate balance between control that, as you rightly point out, often leads to greed, and the anarchy of too open a system. My guess is that the inter-operable web won't completely fail but won't fully realize the utopian dreams that many espouse.

Beyond the greed factor is simple laziness. Conforming to standards simply requires more work and more education on the part of the players. And standards are necessary to the semantic web.

Too many people won't bother. "The site works on my browser or for Windows or whatever. That means it's good enough."

There is a simple test that illustrates this point. View the HTML of any page at random. Chances are that well over 90% of sites don't come anywhere close to complying with published web standards. If you can't get people to follow what are comparatively simple rules such as W3C standards, how can you expect them to follow the more complicated API's that inter-operability require?

Rereading, I now realize my last post sounds somewhat left-field in context with the other poster's comments... so let me add a bit.

If you grok my vision of the future, you'll see that what I propose is that individuals will eventually control most of whatever information ends up exposed on the semantic web.

If I control my data, in the same way that I control my identity (which finally seems to be happening, even for people who rely on the services of others to host their data rather than literally running their own web servers), then I control who can see it and how, just as well.

As a consumer making rational decisions based on the marketplace, I will decide what services I use in the future, just as I decide what programs I install on my PC today; I will decide which webapps I will use; I will decide which storage service to use; I will decide which identity service I'll use.

And if any company wants my business, it had better perform at least as well as the free (as in speech, and maybe as in beer, if we're lucky) alternatives. I will not sit still and let my essence, my data, be usurped, nor will I put up with a crappy closed ecosystem when there will be a perfectly serviceable (but probably chaotic) open alternative to it built by hobbyists -- people who do what they do for love (of what they do), not just money (though a little money never hurts).

The same people who build intricate domain-specific blogs because they love the subject will see the semantic web as an extension of that and they will flock to it just as soon as it's ready for prime time -- or, probably, even earlier.

That will be the beginning. Once people begin to crosslink anything and start to see the benefits in action, the dominoes will star to fall. If Google or MySpace or whoever doesn't let me share what I want to share, I will take my business elsewhere... and you can be sure that at some point, at least some of these players (or new ones) are really going to get that.

The semantic web is inevitable. And like so much else, it's rollout -- when it comes -- will happen on Internet time. (Please forgive the cliché.)

I look forward to the day when your research agent finds this post for you while you're doing a research paper in the future (maybe circa 2010? 2015?), and manages to figure out who I am and how to reach my secretarial agent, and the two of them put us in touch. We'll do lunch. ;-)

An example (supporting “Anonymous) of with natural language semantics at work is at Boston’s Children’s Hospital at their Center for Medial and Child Health www.cmch.tv/research/ . They have made some progress there using the Semantxls “smart search” engine that spans understanding (ontology-based) of ten vocabularies from “social science” professions. Results are filtered and ordered more precisely when using lots of contextual “jargon” in well-articulated local community of practice-specific vocabulary. You'll not need all of the machinery associated with the "semantic web."

Who are you again? I'll take Zeldman's prediction of the Semantic Web over yours any day of the week. His real world experience with companies corporate and small-business paint a totally different picture, with much more breadth and depth than a single comment can reveal.

Your points are as valid for commercial as open source projects. The separate "tribes" which make up each group is incapable of cooperating long enough to achieve common goals.

One only needs to look at the way the open source community is tearing itself apart over the new version of the GPL to see that they are incapable of putting their "tribal" biases aside long enough to achieve a OSC-wide initiative, let alone an internet-wide initiative. Expecting commercial entities to do any better is unrealistic.

Your comment that the web protocols in use today emerged out of government and academia are, of course, historically correct. Commercial input only came after it was a fait accompli and commercial entities are not wired to work toward the common good. They have shareholders to satisfy and that group is incapable of thinking beyond the individual good. If it were different, major social and ecological problems would already have been solved through common effort.

I have always viewed web 2.0 and the pending web 3.0 as mere marketing hype. None of the comments from others in response to your post has changed by mind.

Hi Stephen, I'm glad you've brought this up, even though I completely disagree with your prediction...

I agree with janne, Tim and some of the other commentators on their technical points, but I don't think it's necessary to go there to counter your arguments.

Ok, I reckon a distrust of business in general is well-founded, because the primary motivation is self-interest. But businesses do cooperate, when they see it as being in their own interests. In fact commerce can only function when businesses work together at their interfaces. Money is a shared vocabulary with a set of standard protocols.

The web has been hugely successful. A factor in its success must be its distributed nature - the system as a whole is, in principle at least, open to anyone. It's hard to see how any single group or organisation could break it, though the threat to things like net neutrality and the intervention of governments are worrying. Overall the web has oiled the economic wheels, led to significant profit for those companies that have embraced it. The current web works despite the proportion of backstabbing money-grubbing leeches.

The Semantic Web is an extension of the current Web, essentially making more use of first-class data alongside the human-readable documents. Increased connectivity between systems goes hand in hand with this, along with the extra potential for automation it enables. But I don't see a difference here that would reverse the position of existing commercial interests. Things like the Yahoo!/Flickr login issue are sand in the cogs, but as well as affecting the end user, they will also have some impact on the companies involved. Swimming against the flow is expensive.

I'd be very interested to hear more about what you call personal computing. I've a strong suspicion it may be close to a facet of what I'd call the Semantic Web ;-)

I fully agree with the post and to the existing comments I'd like to add:

semantic web = plumbing?

Only if you see the future consisting of web-apps being VisualBasic style clicked-together blobs. Personally, I don't want that!

Corporations making significant investments in the current semantic web?

Yes they do, its called SEO. ;-)

The semantic web vision is inevitable.

Vision and reality are two different pair of shoes. Take RSS content for example. As a protocol to keep you updated of your favourite blog it might work. Otherwise it sucks. Do a deep RSS search and you'll find that half of it is SPAM and the other half plagiarism.

The 'web of data'.

To manually set this up is too much work. Hear me - WAY too much work, especially if it is to be free. Why would anyone add meta information to add semantics unless there is a benefit. Never mind the issues around protocol and 'what is true'.

Longevity of Web2.0 APIs

Google let's us search because we click on the Ads which in turn feeds the cow. If we'd all stop clicking, nobody would be searching on Google in short order. And let's not forget many of these 2.0 companies are running on VC capital; risky investments in search for the right 'exit strategy'.

Your story rings true for me. I've been involved in "collaborations" on standards since the mid-80's (pre-Tcp/ip) and have seen the majority of those efforts founder for the reasons you enunciate. I'm in the middle of a project that's a no-brainer for realizing the benefits of collaboration. Yet we are in our 6th year of trying to get IBM, Yahoo!, Microsoft and Oracle to work together on a common standard, or even to sit at the same table at the same time. The market we seek server is relatively small, troublesome, and fragmented. There's no commercial incentive to work together: it will never be worth the effort. However, every single person we've engaged believes that the effort is worth pursuing for social, political and humanitarian reasons. Just not together.

I suspect the resistance you're getting is from people who don't have much experience in this area. It's always much easier to imagine utopia than acheive it.

Great blog, and that's exactly why I created www.open4.org as an open source social networking site - because every other place I wanted to network either restricted what I could say or do on their site, or wanted me to pay somewhere along the line.

As soon as $$$ come into the equation, that's all that counts, which is why GPL stuff works in the end, because it's about the software, not the dollars.

I suppose that you're not familiar with Axelrod's The Evolution of Cooperation which convincingly argues how cooperation can emerge particularly in cases where agents have opposing interests. Provided that anybody can make money off the semantic web, it will happen. I am doing some research into these kind of ideas, coincidentally the kind that the NRC goes out of it's way to not fund.

IMHO, Web 2.0 represents the efforts to synergise the web 1.0 applications and then throw a liberal sprinkling of the "social" thing. The true semantic web, will need to do more than marry an email id with a set of snaps and with time/place shifted TV viewing. It will need to enable machine reasoning - beyond threading well-known identities / identifiers.

Indeed this (= going beyond identity threading) is not/will not be necessarily encouraged by the yahoos and googles of today. But this will be pursued and created by the yahoos and googles of tomorrow.

Remember when search engines would read and pay attention to Meta tags? You know, when anyone could 'tag' their pages (or data) and tell the rest of the web what it was? Remember how well that worked? Yeah. The search engines were forced to abandon Meta-tags when they realized that every page on the web was about sex or viagra or some other popular and lucrative term.

Semantic Geeks, get real. If there is value to be had in sharing data with the world then many people (and virtually all companies) will share it in such a way as to maximize that value and bring as much traffic to themselves as possible. Case in point: the aforementioned SEO.

Furthermore, fully shared data will only remain shared and usable by others so long as the sharer is getting more out of the deal than the sharees. For example: Say Viacom puts all their video online in a searchable database with complete metadata. Then Google Video picks it all up and, since they have a better site/interface/brand than Viacom, attracts a bigger audience for those videos than Viacom. What happens next? The videos get yanked and, perhaps, someone gets sued...

Interesting viewpoint. However I'm one of those people who believe that the semantic web is an effect not a cause; i.e. it is the result of people wanting to cooperate and not finding a suitable set of standards to do so.

Anyone who has done any kind of work connecting businesses electronically will tell you that a set of standards describing data, is welcome and eventually inevitable. Its probably not going to be the silver bullet of integration, but I think it will solve a lot of problems and create a lot of oppurtunities for sharing data meaningfully.

You're right about cut throat business practices - that is how a lot of businesses survive the free market economy. However businesses also survive because they cooperate with the right partners. So while the semantic web may not be a universal cure-all just yet, it will definitely provide a platform for existing cooperative models to function better.

I don't think your views are unfounded, and though you describe your viewpoint as non-technical I think it is cause for thought. However I'm more optimistic about about the semantic web, because it provides standards and encourages co-operation - 2 factors which history has repeatedly shown to greatly accelarate progress.

Yeah, this whole semantic web movement is smoking crack in my opinion. I went to a seminar at Stanford 2 years ago. All the guys who showed up were washed Artificial Intelligence guys – oh by the way, AI was “the holy grail” in the late 1980s when people discovered (this was before crack) that AI guys were smoking pot. Then AI made its way into different technologies and today lives in the form of linguistics, latent semantic analysis, machine learning etc. Anyways, these AI guys in a self serving way want the entire world to define RDF files for all the different domains in the world, go through every website make in semantic web compliant. I was told to study this at Infospace by my boss and after 1 year researching this and came to the conclusion à My boss and these AI guys were crackheads.

The W3C is a good organization in theory, but in reality they've had their heads up their arses too long. A good idea comes along, their implementation falls short, and they refuse listen to anyone, dragging standards development out for years. Case in point - if the X in XHTML stands for eXtensible, why can't I declare and use a <foo bar="1"/> tag in my document? Why do I have to use <p></p> instead of the perfectly valid <p/> in order for my HTML to validate? They tell us the table tag is dead, but have yet to offer a valid div/CSS alternative. The box model is flawed because the box grows when you add padding inside it, and you can't even use vertical-align:middle to align things vertically inside a div. I've been building websites for over ten years and there have never been real standards. One browser or the other refuses to implement a flawed part of their standards, so then they move on to the next version, and make the same sort of mistakes. Rinse, repeat.

The web is bigger than google, yahoo and flickr. Call it what you will, but the web will continue to evolve.It seems like a natural progression for machines to eventually be able to use the data on the web just like people do today.

The web has to reflect more than just semantic memory. This is why a semantic web will not 'work'. Because people organize experience using more than semantics, a semantic web fails to account for the additional organizational structures upon which semantics are founded.

If we observe the internet technology and economy, the innovation, and what becomes standard, comes from struggling startups.

MySpace, YouTube, Google, Yahoo, etc. innovated when they were startups. Startups luckily do not have an option but to innovate and create new business rules and markets that create an advantage for themselves.

The Big Players behave to retain the advantange that they have. They can innovate too for the same reason but most often the status quo is comfortable and advantageous at the same time.

I would think that basing the success of semantic web and web 2.0 collaboration completely on big corporates is incorrect.

You are not taking into account the chance that there will always be smaller players who will push for an evolution towards cooperation, if that is the only way they can gain an advantage in the market.

If that is what appeals the consumers the big corporates will have a tough time working against this force of cooperation.

It will be interesting to analyze the internet business and technology on the basis of game theory, where the players are individual consumers,small startups and big corporates and try to find out the Nash equilibrium(a set of strategies that is beneficial to all parties) that is prevailing or will prevail.

Here is one quick analysis.

Lets say the players are corporates who want to make money, the startups who want to make money, the individual consumer who wants to spend money but get good value for that money.

I am assuming It is safe to say that every individual consumer wants to consume services that are seamless and work effortlessly with other services.

A small example: I want my photo to be available to me in the camera, to my wife on my home pc and to my family over a internet site as easily and seamlessly as possible.

Don't you think that the corporates and startups can make money only if they make this possible.Same example: I upload the photo to my PC and organize them using picasa, which also helps me seamlessly upload it to snapfish which is a different business entity than google. I ended up using picasa as a photo manager because that's the feature I like.

Similarly for other services I am predicting the same behaviour, meaning seamless collabaration will become an important characteristic of every internet service , because it will become more lucrative than non-cooperative models.

I think that the semantic web will become a reality, but it will require a bit of self-synchronization by industry, based on what appears to be a good system of interaction. The same route has been taken again and again through the history of commerce. Some details and a better worded argument than I present here are at: Why the Semantic Web Will Not Fail

"We are talking about the most conservative bunch of people in the world, people who believe in greed and cut-throat business ethics. People who would steal one another's property if it weren't nailed down. People like, well, Conrad Black and Rupert Murdoch."

Stephen, if you start from the premise that business leaders and the world's economic system is inherently dishonest and cut-throat, you've got a lot more to worry about than the semantic web. I am amazed so many people got sucked into this tirade.

Stephen, if you start from the premise that business leaders and the world's economic system is inherently dishonest and cut-throat, you've got a lot more to worry about than the semantic web.

Maybe if I hadn't spent the last decade chronicling their behaviour on the internet I might have a more charitable view.

But honestly, every day it seems like there's more evidence that I'm right and that the charitable view is wrong. Yesterday's OLDaily, for example, mentions the sign-on issue, some company promoting itself using the word 'apartheid' (on which it has slapped a service mark), and the ongoing scam that is 'education research' being promoted by certain companies in the U.S. That's one day. And every day is like that.

When I make predictions, I assume businesses will behave badly. My predictive record is very good. If my assumption were false, it would impair my predictions. It hasn't, so...

The semantic web is based on a brain-dead epistemological model. There are just things you can't point to with an URI, for example the interpretations of an radical islamist of the United States. It might bring "value" to certain tasks when we work in unison on certain repeated problems, but then again standards are good as long as they are simple to implement by any developer... and what I have seen so far of semantic web, it's not.

The blog is quite right. As an AI guy, let me give you a couple of more technical reasons why I don't like the whole SW hype:

First and foremost, people in AI have worked on formal logics (on which most SW standards are based) for a very long time. Remember Prolog, LISP and knowledge based systems. etc ? They were interesting to play around with, but they all failed. SW technologies are really NOT, I repeat NOT more advanced than such systems in the underlying logical inference they use.

Why, you would ask, would anybody think these techniques would work this time? The big answer, they will quickly point out, is the Web which, supposedly, changes everything. What they mean by that is that every guy that design webpages suddenly becomes a logician (and not any type of logician, one that uses exactly the pre-defined standard they though of). Actually, the web makes it even more difficult to meaningfully apply such formal approaches. I mean, if you can't persuade people to use logic-based rules to specify such systems and get them to do useful things in very closed, controled environments, why would anybody do it in a large, open environment? (by "anybody" I really mean "anybody" who doesn't get big bucks research subsidies, to supposedly do something "semantic").

OK, I'm not you, but here's my 2 cents prediction about the future of the SW. It will never take off, that is sure. But will people continue to work on it, organize conference, print mountains of books, journal proceedings etc. ? I bet you! Simply because people who have invested their whole (research) life in this, find it impossible to admit it's all hot air.

They will keep, of course complaining that people who criticize them can't see this is the future of the net, are illiterate or even stupid, have not been enlightened by Tim Berners Lee's books (which, by the way, I greatly respect as the inventor of the net). Kind of like a closed sect, really.

the first statement that goes in your direction was in the german it magazine c't in the last year. where they summarized the whole "we make one for all" never worked. the long tail of the web 2.0 instead is so narrow, and the www technologie so so bad(sic!) that the whole network thing shows that in this world its not about well done like semantic web. good enough like web 2.0 is sufficent.

Any standards that exists in industry are the result of, or attempts at commoditization.

I just picked up a book called "Joel on Software" and it's got a pretty good assessment of compatibility in the industry.

It's not in everyones best interest that the plumbing works. Do you think database companies like mysql and oracle think db abstraction layers are a good idea if the layer allows you to easily switch db vendors? Depends on who's writing it, who has the market share, etc. . . DirectX is a standard - So is HTML, and OpenGL. They exist for different business reasons though.

Do you want the semantic web to fail? How exactly are you defining the term (and Web 2.0, if that needs to enter the conversation)? What if a prototype were created by the open source community? It seems that saying that the semantic web will necessarily fail is like saying that open source cannot work, which is clearly false. I suspect you meant "fail with respect to [something]" -- could you elaborate?

I see the semantic web being started by individuals, such as myself, who can build a relatively small but perhaps scalable version which demonstrates economic value. The key is to make the greatest economic incentive line up with how you want the semantic web to evolve. I'm not convinced this is impossible.

You mention competing vocabularies being an issue. Others have pointed out that the W3C doesn't require an official vocabulary. More importantly, the world doesn't operate on an official vocabulary -- if it did, there would be a lot less misunderstanding! Translation from one set of vocabulary to another will always be required; see multiple languages for an irrefutable example.

I find it splendid how you have managed to engaged so many, that in itself must be worth having to deal with the accusations (about missing the point) and controversies, right? But if you care to read MY 5 cents of an opinion, please head over to http://del.icio.us/torkveen and look at today's postings

Interesting...errmm...BS that is. I have no idea why you keep on referring to services like Google, Flickr and especially Myspace. There is nothing semantic about these (or similar) services.

On the other hand I don´t understand why you refer to companies not being willing to ´network seamlessly´ with each other?! Ain´t that just ludacris. Tranformation to web of systems and eventually to system of systems includes absolutely no requirements to decreased data (or system) hygiene.

Talking about standard? I guess html did pretty well...even in the Corporate world. So did TCP and HTTP...errmmm...let me thing if there is more...but then again, why would I!

According to any and all market reports, Corporations are nowhere near the early adaptor group. Which already is at full-speed doing just that, adapting.

Do you perhaps suppose that web will stay as it is, with no applied semantics. No meanings, no knowledge value. Just exponentially growing quantities of crappy content, long-tail inventories and plethora of services. Sure, that´s really likely. How come I didn´t come to think of that ever?! Ah, it´s cause it makes no sense.

As a hint for the future, it works better for marketing to do posts like ´Why Myspace & Flickr Will Fail´. And to be honest, you seem much more at ´home´ with concepts like that.

There are certainly lots of hurdles posed into the realization of Semantic Web but there is also remarkable adaptation seen from both - industry and academia - for sematic web stuff. There may not be any certainty in terms of timeline about realization of semantic web but the signs indicate that its certainly coming sometime (may be after a long time).

A comprehensive take on Semantic Web and Web 3.0 at http://techbiz.blog.com/1730241/

The criticism of complexity was certainly true of the semantic web standards released in 2004. That complexity combined with a lack of tools is the main reason the semantic web fizzled.

In January of this year the W3C released a working draft for a simplified standard: RDFa. This new standard will enable semantic attributes to be embedded directly into XHTML. A clear introduction to RDFa can be found at http://www.w3.org/TR/xhtml-rdfa-primer/

We believe RDFa, along with a supporting tool set, can form a bridge to the Semantic Web envisioned by Tim Berners-Lee.

Semantic Bridge Technologies (http://www.semanticbridgetechnologies.com) is a startup company located in Austin, TX. We are creating a tool set and the supporting infrastructure for the implementation of the Semantic Web. We are taking a very pragmatic approach. Our target audience is comprised of web designers and software engineers who build Internet applications not theorists who study semantic structures. We are building a bridge, not an ivory tower.

One of the key aspects of the Semantic Bridge Project is the creation of a dynamic and interactive ontology management system, “The Semantic Knowledge Repository”. This system along with the tools that will allow web designers and software engineers to easily interact with the repository will have a profound impact on the rapid deployment of the Semantic Web.

I'm a developer; I've got a few websites I gain advertising money on. I spent a lot of time setting up these sites to draw in users, framing my information in a positive (and correct) light.

Do I want random weirdos from all over the web creating mashups from my work without having to credit me, refer people to my site, or even show the work in the proper context?

HELL NO.

If you want what I've got, you have to come to MY SITE and get it. You have to give ME your clicks. And what's wrong with that? I earned them by building the site in the FIRST place. As a matter of basic principle, the least courtesy people owe me is to get my stuff from me. Don't you think?

Most of this "mashup" and "semantic web" stuff is pie in the sky dreaming by people who want a free lunch. They want to be able to create their own site (with advertising revenue! Hah!) by pilfering data from other sites. They want to access people's data and output without even having to visit their site. And nobody will EVER agree to this once they figure it out.

What's really funny about it is, you have to do all kinds of extra work to make it possible for this to happen in the FIRST place.

The blogger is absolutely right; the Semantic Web is dead meat.

Oh, and before anyone calls me a greedy corporate suit, understand that I'm strictly small-business, a regular joe. But I think that if you build something, you should be able to benefit from it without having the whole web try to cash in for free.

To all those above who are saying "you're wrong," the fact of the matter is that Stephen is right a lot of the time. Like you, I disagree with his assessment about the inevitable failure of the Semantic Web, mainly because I want to believe in Tim B-L's vision, not because I think I can prognosticate. Those who claim to know better, have you tried writing a SPARQL query yet? I think what SD really wants is to spark conversations (which he has clearly done here).

Sorry for this late comment, I was asked to research the semantic web recently.

In short, I agree that the semantic web isn't going anywhere and is one more utopian pipedream.

If only people would be perfect and add perfect tags and be perfectly honest, then the world wide web would be a much better place.

Righttttt. With the exception of Carmax, there is NO automobile dealership that has the least interest in people actually comparing prices. Nada.

And there is one more beef I have with metadata gurus and their disciples. They don't seem to think creating the original data in the first place counts for much.

It is much more fun to bust on people for not adhering to standards and formats and peer review and editorial review and source citations and writing style ...

Look at Wikipedia and how many articles are now tagged as substandard. Think any of those taggers ever thought to get off their ass to improve the article? Hell no ... easier to sit back and criticize.

Some sort of index/ metadata/ organization is required to search and access stuff. But there is nothing magical about the index. If it were destroyed, a new one can be created (maybe with a lot of work). If the ORIGINAL DATA went missing, you are really up the creek and you probably will never be able to recreate the original.

One last thought. We don't need any more "one off" layers in the internet. When I do research, I want the original source no matter how much crap I have to wade through in a search to get to it.

The web is already the most incredible research tool I have ever used. A search that took days, weeks, years ... now in minutes.

And if I have to look at 200 irrelevent hits? So be it.

If the semantic web was going to succeed, Google, search engines, and everyone would have latched on to it right away.

the 'semantic web' as its envisioned as a w3 standard 'knowledge-base'is ill-conceived and will be shown to be useless because it fundamentally does not work/emulate the way knowledge works.

"to know" is to construct one's own design of meaning conceptual network from/to a social network of shared meaning. if the meaning is fixed in the standard without regard to one's own disgressionary construct process, it can not be assimilated/learned and will not become/contribute to one's knowledge- (in other words; impossible to be what the standard intends to be).

however, the web itself will continue to refine as a robust, general information repository due to the participation of all who use, add to, and manipulate the combined information it encompasses.

I reckon that when there is a way to make money out of Semantic web, that's where the standards will be laid down. From a commerical side we'll be paying for information from startups that were free to sign up to. The semantic web as a useful tool where we search and build up our own view of the world will rely on trusted sources, and we'll be either paying a premium for that trust or spending our time filtering through spam. The issue I guess is that the technology hasn't matured (simplified) enough to make it into widespread use (creation of suitable data). For example, can this textbox be changed into a semantic data entry field? Until there is widespread available semanticly tagged information there will be no useful services built upon that data. Give it an infinite timeline, it might work.

That is just an intuition supported by a narrow minded thinking. The technological achievements may not be changing the human characteristics but the meaning of the business by which humans are making profit is surely changing.

Most of these cut-throat bosses are going to lose their beloved sector positions. The cause of this situation is not the Semantic Web but the semantics behind the semantic web.

You are probably right, in saying the Semantic web will fail. But I guess by that you mean only the current approach. There are a number of other approaches which are running parallel, if one fails we might take another approach. that is how engineering had been, and that is how human race came up to this. For example look into Semandeks http://semandeks.com It is not dependent on any formats. at the same time if any final format emerges it can quickly adapt to it.

This is an interesting blog post. Are you aware of what the US Securities and Exchange Commission (SEC) is doing with XBRL? In fact, financial reporting globally is knocking on the door of the Semantic Web. (See more information here http://xbrl.sec.gov/)

Basically, every public company which reports to the SEC will be doing so in XBRL by July 2011, it is already the law. You may, or may not, consider XBRL a Semantic Web technology. I think this because you can convert the XBRL syntax which expresses business reporting semantics into RDF/OWL. There is a long way to go, but the US SEC is helping to prime the Semantic Web pump.

I think the Semantic Web will be here sooner than most people realize. I could be wrong. It will be interesting as things evolve over the next five years or so.

While the ideals of the Sem Web are great, to gain mass support for it to work, it requires one key thing. This is the sem webs ability to be sucked up fast by techie geeks so they can provide the next killer services using the sem web and stop relying on publishing their data with "open" api's. The web service stack, I know most developers don't like to pick it up, because it is a shameful attempt at making a standard that has resulted in a bloated system that is horrible to even think about. The web service stack was never picked up outside of companies who wanted to integrate systems. I hope the sem web doesn't go that route.

Also, the benefits of the sem web are philosophically right and not always monetarily. I think this is the crux of this blog post. The sem web is going to clash with big companies unless these companies figure out ways in which to use the semantic web for their advantage. I think facebook will be against the sem web, their business will not benefit from open data (but this is arguable, because facebook already tries to share some of their data, why not use the sem web to do this?). Google on the other hand, imagine an artificially intelligent search engine, why wouldn't google promote the sem web???