Tools

by
Kumiko Tanaka-ishii, Satoshi Tezuka
- In Annual Meeting of the ACL, 2005

"... This article presents a novel approach for readability assessment through sorting. A comparator that judges the relative readability between two texts is generated through machine learning, and a given set of texts is sorted by this comparator. Our proposal is advantageous because it solves the prob ..."

This article presents a novel approach for readability assessment through sorting. A comparator that judges the relative readability between two texts is generated through machine learning, and a given set of texts is sorted by this comparator. Our proposal is advantageous because it solves the problem of a lack of training data, because the construction of the comparator only requires training data annotated with two reading levels. The proposed method is compared with regression methods and a state-of-the art classification method. Moreover, we present our application, called Terrace, which retrieves texts with readability similar to that of a given input text. 1.

...r V ab is 1, that for V ba will be –1. In terms of constructing a feature vector V a for a text a, substantial features have been proposed (Klare 1963; DuBay 2004a, 2004b; Schwarm and Ostendorf 2005; =-=Pitler and Nenkova 2008-=-). In this work, we only utilize the most basic features of vocabulary in terms of word frequencies for three reasons. First, as stated in Section 2, because the focus of this article is not to study ...

by
Chenhao Tan, Evgeniy Gabrilovich, Bo Pang
- In Proceedings of the 5th ACM International Conference on Web Search and Data Mining, 2012

"... Imagine a physician and a patient doing a search on antibiotic resistance. Or a chess amateur and a grandmaster conducting a search on Alekhine’s Defence. Although the topic is the same, arguably the two users in each case will satisfy their information needs with very different texts. Yet today sea ..."

Imagine a physician and a patient doing a search on antibiotic resistance. Or a chess amateur and a grandmaster conducting a search on Alekhine’s Defence. Although the topic is the same, arguably the two users in each case will satisfy their information needs with very different texts. Yet today search engines mostly adopt the onesize-fits-all solution, where personalization is restricted to topical preference. We found that users do not uniformly prefer simple texts, and that the text comprehensibility level should match the user’s level of preparedness. Consequently, we propose to model the comprehensibility of texts as well as the users ’ reading proficiency in order to better explain how different users choose content for further exploration. We also model topic-specific reading proficiency, which allows us to better explain why a physician might choose to read sophisticated medical articles yet simple descriptions of SLR cameras. We explore different ways to build user profiles, and use collaborative filtering techniques to overcome data sparsity. We conducted experiments on large-scale datasets from a major Web search engine and a community question answering forum. Our findings confirm that explicitly modeling text comprehensibility can significantly improve content ranking (search results or answers, respectively).

"... The goal of DARPA’s Machine Reading (MR) program is nothing less than making the world’s natural language corpora available for formal processing. Most text processing research has focused on locating mission-relevant text (information retrieval) and on techniques for enriching text by transforming ..."

The goal of DARPA’s Machine Reading (MR) program is nothing less than making the world’s natural language corpora available for formal processing. Most text processing research has focused on locating mission-relevant text (information retrieval) and on techniques for enriching text by transforming it to other forms of text (translation, summarization) – always for use by humans. In contrast, MR will make knowledge contained in text available in forms that machines can use for automated processing. This will be done with little human intervention. Machines will learn to read from a few examples and they will read to learn what they need in order to answer questions or perform some reasoning task. Three independent Reading Teams are building universal text engines which will capture knowledge from naturally occurring text and transform it into the formal representations used by Artificial Intelligence. An Evaluation Team is selecting and annotating text corpora with task domain concepts, creating model reasoning systems with which the reading systems will interact, and establishing question-answer sets and evaluation protocols to measure progress toward this goal. We describe development of the MR evaluation framework, including test protocols, linguistic resources and technical infrastructure. 1. Background and Conceptual Framework To advance research towards the end goal of general, lightly-trained systems which read text “in the wild ” well enough to support automated reasoning tasks as well as to

...as sentence length or syllable count (Flesch, 1948; Kincaid, et.al., 1975), although some incorporate deeper concepts, such as word or sentence “complexity” (Gunning, 1952). However, recent research (=-=Pitler & Nenkova, 2008-=-) has shown that surface features are not well correlated with judgments by adult readers, whereas several syntactic, semantic, and discourse features are. Reflecting the goal of the MR program, we de...

"... We present a new algorithm to measure domain-specific readability. It iteratively computes the readability of domain-specific resources based on the difficulty of domain-specific concepts and vice versa, in a style reminiscent of other bipartite graph algorithms such as Hyperlink-Induced Topic Searc ..."

We present a new algorithm to measure domain-specific readability. It iteratively computes the readability of domain-specific resources based on the difficulty of domain-specific concepts and vice versa, in a style reminiscent of other bipartite graph algorithms such as Hyperlink-Induced Topic Search (HITS) and the Stochastic Approach for Link-Structure Analysis (SALSA). While simple, our algorithm outperforms standard heuristic measures and remains competitive among supervised-learning approaches. Moreover, it is less domain-dependent and portable across domains as it does not rely on an annotated corpus or expensive expert knowledge that supervised or domain-specific methods require.

... kNN classifier are interpolated with the ones from the SVM classifier to produce a final prediction, which is better than using either one of the classifiers alone. Most recently, Pitler and Nenkova =-=[21]-=- examine by far the largest set of textual features. Their feature set includes word (unigram language model), syntactic (identical to the parse features in Schwarm and Ostendorf’s work [22]), lexical...

"... This paper present a new readability formula for French as a foreign language (FFL), which relies on 46 textual features representative of the lexical, syntactic, and semantic levels as well as some of the specificities of the FFL context. We report comparisons between several techniques for feature ..."

This paper present a new readability formula for French as a foreign language (FFL), which relies on 46 textual features representative of the lexical, syntactic, and semantic levels as well as some of the specificities of the FFL context. We report comparisons between several techniques for feature selection and various learning algorithms. Our best model, based on support vector machines (SVM), significantly outperforms previous FFL formulas. We also found that semantic features behave poorly in our case, in contrast with some previous readability studies on English as a first language. 1

...r (1983), eventually admitted not being able to demonstrate the superiority of those new predictors over traditional ones. More recent work also reported limited evidence of this alleged superiority (=-=Pitler and Nenkova, 2008-=-; Feng et al., 2010). In order to clarify as much as possible the situation for FFL, we implemented the following features: Personnalization level: Dale and Tyler (1934) suggested that informal texts ...

"... Great writing is rare and highly admired. Readers seek out articles that are beautifully written, informative and entertaining. Yet information-access technologies lack capabilities for predicting article quality at this level. In this paper we present first experiments on article quality prediction ..."

Great writing is rare and highly admired. Readers seek out articles that are beautifully written, informative and entertaining. Yet information-access technologies lack capabilities for predicting article quality at this level. In this paper we present first experiments on article quality prediction in the science journalism domain. We introduce a corpus of great pieces of science journalism, along with typical articles from the genre. We implement features to capture aspects of great writing, including surprising, visual and emotional content, as well as general features related to discourse organization and sentence structure. We show that the distinction between great and typical articles can be detected fairly accurately, and that the entire spectrum of our features contribute to the distinction. 1

"... Most work on automatic text simplification considers lexical difficulty separate from syntactic simplification. In this study, we use both factors together to predict a variety of sentence changes, including the standard problems of splitting and short-ening, as well as expanding to define difficult ..."

Most work on automatic text simplification considers lexical difficulty separate from syntactic simplification. In this study, we use both factors together to predict a variety of sentence changes, including the standard problems of splitting and short-ening, as well as expanding to define difficult words that are im-portant to the topic. We leverage a variety of lexical and parse features, as well as a score of the relatedness of a sentence to the topic of its document. Index Terms: text simplification, parse complexity 1.

...es, seem promising, there is still room for improvement in characterizing the conditions in which splits and expansions take place, including looking at features from more general work on readability =-=[25]-=-. Previous work and our own analysis indicated that three common changes in simplification are replacing or explaining difficult words and phrases, removing extraneous details, and separating syntacti...

"... Readability formulas are methods used to match texts with the readers ’ reading level. Several methodological paradigms have previously been investigated in the field. The most popular paradigm dates several decades back and gave rise to well known readability formulas such as the Flesch formula (am ..."

Readability formulas are methods used to match texts with the readers ’ reading level. Several methodological paradigms have previously been investigated in the field. The most popular paradigm dates several decades back and gave rise to well known readability formulas such as the Flesch formula (among several others). This paper compares this approach (henceforth ”classic”) with an emerging paradigm which uses sophisticated NLPenabled features and machine learning techniques. Our experiments, carried on a corpus of texts for French as a foreign language, yield four main results: (1) the new readability formula performed better than the “classic ” formula; (2) “non-classic ” features were slightly more informative than “classic ” features; (3) modern machine learning algorithms did not improve the explanatory power of our readability model, but allowed to better classify new observations; and (4) combining “classic” and “non-classic ” features resulted in a significant gain in performance. 1

"... Abstract—Reading is an integral part of educational development; however, it is frustrating for people who struggle to understand (are not motivated to read, respectively) text documents that are beyond (below, respectively) their readability levels. Finding appropriate reading materials, with or wi ..."

Abstract—Reading is an integral part of educational development; however, it is frustrating for people who struggle to understand (are not motivated to read, respectively) text documents that are beyond (below, respectively) their readability levels. Finding appropriate reading materials, with or without first scanning through their contents, is a challenge, since there are tremendous amount of documents these days and a clear majority of them are not tagged with their readability levels. Even though existing readability assessment tools determine readability levels of text documents, they analyze solely the lexical, syntactic, and/or semantic properties of a document, which are neither fully-automated, generalized, nor well-defined and are mostly based on observations. To advance the current readability analysis technique, we propose a robust, fully-automated readability analyzer, denoted ReadAid, which employs support vector machines to combine features from the US Curriculum and College Board, traditional readability measures, and the author(s) and subject area(s) of a text document d to assess the readability level of d. ReadAid can be applied for (i) filtering documents (retrieved in response to a web query) of a particular readability level, (ii) determining the readability levels of digitalized text documents, such as book chapters, magazine articles, and news stories, or (iii) dynamically analyzing, in real time, the grade level of a text document being created. The novelty of ReadAid lies on using authorship, subject areas, and academic concepts and grammatical constructions extracted from the US Curriculum to determine the readability level of a text document. Experimental results show that ReadAid is highly effective and outperforms existing state-of-the-art readability assessment tools. I.

...ing readability analysis methods, ReadAid applies new, sophisticated measures to assess the readability level of a document beyond its syntactic and semantic levels. Kate et al. [3] and Pitler et al. =-=[8]-=- employ a regression analysis algorithm that combines a diverse set of syntactic, lexical, and language-model based features, such as the number of pronouns, length of sentence, etc., to determine the...

"... Sentence fluency is an important component of overall text readability but few studies in natural language processing have sought to understand the factors that define it. We report the results of an initial study into the predictive power of surface syntactic statistics for the task; we use fluency ..."

Sentence fluency is an important component of overall text readability but few studies in natural language processing have sought to understand the factors that define it. We report the results of an initial study into the predictive power of surface syntactic statistics for the task; we use fluency assessments done for the purpose of evaluating machine translation. We find that these features are weakly but significantly correlated with fluency. Machine and human translations can be distinguished with accuracy over 80%. The performance of pairwise comparison of fluency is also very high—over 90 % for a multi-layer perceptron classifier. We also test the hypothesis that the learned models capture general fluency properties applicable to human-written text. The results do not support this hypothesis: prediction accuracy on the new data is only 57%. This finding suggests that developing a dedicated, task-independent corpus of fluency judgments will be beneficial for further investigations of the problem. 1

...? To answer this question, we performed an additional experiment on 30 Wall Street Journal articles from the Penn Treebank that were previously used in experiments for assessing overall text quality (=-=Pitler and Nenkova, 2008-=-). The articles were chosen at random and comprised a total of 290 sentences. One human assessor was asked to read each sentence and mark the ones that seemed disfluent because they were hard to compr...