Capturing semantics using a link analysis based concept extractor approach

by Kulkarni, Swarnim

Abstract (Summary)

The web contains a massive amount of information and is continuously growing every
day. Extracting information that is relevant to a user is an uphill task. Search engines such as Google TM, Yahoo! TM have made the task a lot easier and have indeed made people much more "smarter". However, most of the existing search engines still rely on the traditional keyword-based searching techniques i.e. returning documents that contain the keywords in the query. They do not take the associated semantics into consideration. To incorporate semantics into search, one could proceed in at least two ways. Firstly, we could plunge into the world of "Semantic Web", where the information is represented in
formal formats such as RDF, N3 etc which can effectively capture the associated semantics
in the documents. Secondly, we could try to explore a new semantic world in the existing
structure of World Wide Web (WWW). While the first approach can be very effective when
semantic information is available in RDF/N3 formats, for many web pages such information
is not readily available. This is why we consider the second approach in this work.
In this work, we attempt to capture the semantics associated with a query by rst
extracting the concepts relevant to the query. For this purpose, we propose a novel Link Analysis based Concept Extractor (LACE) that extract the concepts associated with the
query by exploiting the meta data of a web page. Next, we propose a method to determine
relationships between a query and its extracted concepts. Finally, we show how LACE can be used to compute a statistical measure of semantic similarity between concepts. At each step, we evaluate our approach by comparison with other existing techniques (on benchmark data sets, when available) and show that our results are competitive with existing state of the art results or even outperform them.