Conferences

Our goal is to combine the rich multistep inference of symbolic logical
reasoning with the generalization capabilities of neural networks. We are
particularly interested in complex reasoning about entities and relations in
text and large-scale knowledge bases (KBs). Neelakantan et al. (2015) use RNNs
to compose the distributed semantics of multi-hop paths in KBs; however for
multiple reasons, the approach lacks accuracy and practicality. This paper
proposes three significant modeling advances: (1) we learn to jointly reason
about relations, entities, and entity-types; (2) we use neural attention
modeling to incorporate multiple paths; (3) we learn to share strength in a
single RNN that represents logical composition across all relations. On a
largescale Freebase+ClueWeb prediction task, we achieve 25% error reduction,
and a 53% error reduction on sparse relations due to shared strength. On chains
of reasoning in WordNet we reduce error in mean quantile by 84% versus previous
state-of-the-art.

Continuous space word embeddings learned from large, unstructured corpora have been shown to be effective at capturing semantic regularities in language. In this paper we replace LDA's parameterization of "topics" as categorical distributions over opaque word types with multivariate Gaussian distributions on the embedding space. This encourages the model to group words that are a-priori known to be semantically related into topics. To perform inference, we introduce a fast collapsed Gibbs sampling algorithm based on Cholesky decompositions of covariance matrices of the posterior predictive distributions. We further derive a scalable algorithm that draws samples from stale posterior predictive distributions and corrects them with a Metropolis--Hastings step. Using vectors learned from a domain-general corpus (English Wikipedia), we report results on two document collections (20-newsgroups and NIPS). Qualitatively, Gaussian LDA infers different (but still very sensible) topics relative to standard LDA. Quantitatively, our technique outperforms existing models at dealing with OOV words in held-out documents.

Relation extraction is one of the core challenges in automated knowledge base construction. One line of approach for relation extraction is to perform multi-hop reasoning on the paths connecting an entity pair to infer new relations. While these methods have been successfully applied for knowledge base completion, they do not utilize the entity or the entity type information to make predictions. In this work, we incorporate selectional preferences, i.e., relations enforce constraints on the allowed entity types for the candidate entities, to multi-hop relation extraction by including entity type information. We achieve a 17:67% improvement in MAP score in a relation extractiontask when compared to a method that does not use entity type information.

Miscellaneous

06/29/2016: I gave a talk in our reading group about context dependent fine-grained entity typing. pdf

10/07/2014 (While at CMU): I gave a talk on Dirichlet Process and related gaussian mixture models and also introduced a faster inference technique using Cholesky decomposition of scaled covariance matrices. pdf
Edit: I wrote a more detailed note here.