Article Structure

Abstract

A typical knowledge-based question answering (KB-QA) system faces two challenges: one is to transform natural language questions into their meaning representations (MRs); the other is to retrieve answers from knowledge bases (KBs) using generated MRs.

Topics

semantic parsing

Compared to a KB-QA system using a state-of-the-art semantic parser , our method achieves better results.

Page 1, “Abstract”

Most previous systems tackle this task in a cascaded manner: First, the input question is transformed into its meaning representation (MR) by an independent semantic parser (Zettlemoyer and Collins, 2005; Mooney, 2007; Artzi and Zettlemoyer, 2011; Liang et al., 2011; Cai and Yates,

Page 1, “Introduction”

Unlike existing KB-QA systems which treat semantic parsing and answer retrieval as two cascaded tasks, this paper presents a unified framework that can integrate semantic parsing into the question answering procedure directly.

Page 1, “Introduction”

The contributions of this work are twofold: (1) We propose a translation-based KB-QA method that integrates semantic parsing and QA in one unified framework.

Page 1, “Introduction”

Such assumption simplifies the efforts of semantic parsing to the minimum question units, while leaving the capability of handling multiple-relation questions (Figure 1 gives one such example) to the outer CYK—parsing based translation procedure.

Page 3, “Introduction”

Our work intersects with two research directions: semantic parsing and question answering.

Page 6, “Introduction”

Some previous works on semantic parsing (Zelle and Mooney, 1996; Zettlemoyer and Collins, 2005; Wong and Mooney, 2006; Zettlemoyer and Collins, 2007; Wong and Mooney,

Page 6, “Introduction”

In comparison, our method has further advantages: (1) Question answering and semantic parsing are performed in an joint way under a unified framework; (2) A robust method is proposed to map NL questions to their formal triple queries, which trades off the mapping quality by using question patterns and relation expressions in a cascaded way; and (3) We use domain independent feature set which allowing us to use a relatively small number of question-answer pairs to tune model parameters.

Page 7, “Introduction”

Actually, the size of our system’s search space is much smaller than the one of the semantic parser used in the baseline.This is due to the fact that, if triple queries generated by the question translation component cannot derive any answer from KB, we will discard such triple queries directly during the QA procedure.

Page 9, “Introduction”

This paper presents a translation-based KB-QA method that integrates semantic parsing and QA in one unified framework.

Page 9, “Introduction”

Comparing to the baseline system using an independent semantic parser with state-of-the-art performance, we achieve better results on a general domain evaluation set.

feature weights

A linear model is defined over derivations, and minimum error rate training is used to tune feature weights based on a set of question-answer pairs.

Page 1, “Abstract”

Derivations generated during such a translation procedure are modeled by a linear model, and minimum error rate training (MERT) (Och, 2003) is used to tune feature weights based on a set of question-answer pairs.

Page 1, “Introduction”

0 Ai denotes the feature weight of

Page 2, “Introduction”

According to the above description, our KB-QA method can be decomposed into four tasks as: (1) search space generation for H(Q); (2) question translation for transforming question spans into their corresponding formal triples; (3) feature design for and (4) feature weight tuning for {A1}.

Page 2, “Introduction”

2.5 Feature Weight Tuning

Page 6, “Introduction”

Given a set of question-answer pairs {Qh A1?” } as the development (dev) set, we use the minimum error rate training (MERT) (Och, 2003) algorithm to tune the feature weights A11” in our proposed model.

Page 6, “Introduction”

The training criterion is to seek the feature weights that can minimize the accumulated errors of the top-l answer of questions in the dev set:

Page 6, “Introduction”

N is the number of questions in the dev set, Agef is the correct answers as references of the ith question in the dev set, is the top-l answer candidate of the ith question in the dev set based on feature weights AM , Err(-) is the error function which is defined as:

For each entity mention eg 6 Q, we replace it with [Slot] and obtain a pattern string Qpattem (Line 3).

Page 3, “Introduction”

The name of each entity returned equals the input entity mention eg and occurs in some assertions where Q’Ppredicate are the predicates.

Page 3, “Introduction”

Question patterns are collected as follows: First, 5 W queries, which begin with What, Where, Who, When, or Which, are selected from a large scale query log of a commercial search engine; Then, a cleaned entity dictionary is used to annotate each query by replacing all entity mentions it contains with the symbol [Slot].

Page 3, “Introduction”

For each possible entity mention eg 6 Q and a KB predicate ]?

Page 4, “Introduction”

Since named entity recognizers trained on Penn TreeBank usually perform poorly on web queries, We instead use a simple string-match method to detect entity mentions in the question using a cleaned entity dictionary dumped from our KB.

question answering

A typical knowledge-based question answering (KB-QA) system faces two challenges: one is to transform natural language questions into their meaning representations (MRs); the other is to retrieve answers from knowledge bases (KBs) using generated MRs.

Unlike existing KB-QA systems which treat semantic parsing and answer retrieval as two cascaded tasks, this paper presents a unified framework that can integrate semantic parsing into the question answering procedure directly.

Page 1, “Introduction”

Our work intersects with two research directions: semantic parsing and question answering .

Page 6, “Introduction”

In comparison, our method has further advantages: (1) Question answering and semantic parsing are performed in an joint way under a unified framework; (2) A robust method is proposed to map NL questions to their formal triple queries, which trades off the mapping quality by using question patterns and relation expressions in a cascaded way; and (3) We use domain independent feature set which allowing us to use a relatively small number of question-answer pairs to tune model parameters.

Page 7, “Introduction”

This means how to extract high-quality question patterns is worth to be studied for the question answering task.

knowledge bases

Appears in 5 sentences as: knowledge base (2) knowledge bases (3)

In Knowledge-Based Question Answering as Machine Translation

A typical knowledge-based question answering (KB-QA) system faces two challenges: one is to transform natural language questions into their meaning representations (MRs); the other is to retrieve answers from knowledge bases (KBs) using generated MRs.

Compared to their work, our method gains an improvement in two aspects: (1) Instead of using facts extracted using the open IE method, we leverage a large scale, high-quality knowledge base ; (2) We can handle multiple-relation questions, instead of single-relation queries only, based on our translation based KB-QA framework.

Page 7, “Introduction”

(2013) is one of the latest work which has reported QA results based on a large scale, general domain knowledge base (Freebase), we consider their evaluation result on WEBQUESTIONS as our baseline.

Page 7, “Introduction”

As we discussed in the experiment part, how to mine high-quality question patterns is worth further study for the QA task; (ii) We plan to integrate an ad-hoc NER into our KB-QA system to alleviate the entity detection issue; (iii) In fact, our proposed QA framework can be generalized to other intelligence besides knowledge bases as well.

error rate

A linear model is defined over derivations, and minimum error rate training is used to tune feature weights based on a set of question-answer pairs.

Page 1, “Abstract”

Derivations generated during such a translation procedure are modeled by a linear model, and minimum error rate training (MERT) (Och, 2003) is used to tune feature weights based on a set of question-answer pairs.

Page 1, “Introduction”

Given a set of question-answer pairs {Qh A1?” } as the development (dev) set, we use the minimum error rate training (MERT) (Och, 2003) algorithm to tune the feature weights A11” in our proposed model.