Citation and License

BMC Bioinformatics 2012, 13:261
doi:10.1186/1471-2105-13-261

Published: 10 October 2012

Abstract

Background

Semantic similarity measures estimate the similarity between concepts, and play an
important role in many text processing tasks. Approaches to semantic similarity in
the biomedical domain can be roughly divided into knowledge based and distributional
based methods. Knowledge based approaches utilize knowledge sources such as dictionaries,
taxonomies, and semantic networks, and include path finding measures and intrinsic
information content (IC) measures. Distributional measures utilize, in addition to
a knowledge source, the distribution of concepts within a corpus to compute similarity;
these include corpus IC and context vector methods. Prior evaluations of these measures
in the biomedical domain showed that distributional measures outperform knowledge
based path finding methods; but more recent studies suggested that intrinsic IC based
measures exceed the accuracy of distributional approaches. Limitations of previous
evaluations of similarity measures in the biomedical domain include their focus on
the SNOMED CT ontology, and their reliance on small benchmarks not powered to detect
significant differences between measure accuracy. There have been few evaluations
of the relative performance of these measures on other biomedical knowledge sources
such as the UMLS, and on larger, recently developed semantic similarity benchmarks.

Results

We evaluated knowledge based and corpus IC based semantic similarity measures derived
from SNOMED CT, MeSH, and the UMLS on recently developed semantic similarity benchmarks.
Semantic similarity measures based on the UMLS, which contains SNOMED CT and MeSH,
significantly outperformed those based solely on SNOMED CT or MeSH across evaluations.
Intrinsic IC based measures significantly outperformed path-based and distributional
measures. We released all code required to reproduce our results and all tools developed
as part of this study as open source, available under http://code.google.com/p/ytexwebcite. We provide a publicly-accessible web service to compute semantic similarity, available
under http://informatics.med.yale.edu/ytex.web/webcite.

Conclusions

Knowledge based semantic similarity measures are more practical to compute than distributional
measures, as they do not require an external corpus. Furthermore, knowledge based
measures significantly and meaningfully outperformed distributional measures on large
semantic similarity benchmarks, suggesting that they are a practical alternative to
distributional measures. Future evaluations of semantic similarity measures should
utilize benchmarks powered to detect significant differences in measure accuracy.