Second Language Studieshttp://hdl.handle.net/10125/20040
Sat, 10 Dec 2016 03:01:06 GMT2016-12-10T03:01:06ZValidating task-based assessment of L2 pragmatics in interaction using mixed methodshttp://hdl.handle.net/10125/100656
Abstract: This study investigates the validity for task-based assessment of L2 pragmatics in interaction in an English for Academic Purposes (EAP) setting for classroom assessment and meaningful score interpretations for stakeholders. As a validity framework, Kane's (2006) argument-based approach to validity was employed. In view of the complexity of assessing L2 pragmatics in interaction and score interpretations from observed L2 pragmatic performances, score interpretations were built upon interpretive arguments comprising of four inferences, which guided the types of data analyzed and research steps taken in this study. Using the sequential mixed methods approach (Green, 2007; Tashakkori & Teddlie, 2003), qualitative and quantitative evidence was collected to provide valid support for the inferences and to strengthen the validity argument.
Based on a large-scale needs analysis on EAP L2 pragmatic learning needs, two open role-play tasks that are meaningful and relevant to stakeholders in an EAP context were developed. Unlike the closed role-play task format, examinees were allowed to negotiate and naturally interact with interlocutors. In order to provide meaningful score interpretations to the stakeholders and to assist raters in making accurate evaluations of examinees' interaction-involved L2 pragmatic performance, conversation analysis (CA)-informed analytical rating criteria with detailed descriptions were developed based on the detailed analyses of examinees' L2 pragmatic performances in the role-play tasks. One hundred two adult ESL examinees completed the role-play tasks and monologic tasks. Four rater groups, consisting of 12 raters in total, scored each examinee's pragmatic performance. A many-facet Rasch measurement using FACETS (Linacre, 2006) indicated that the role-play tasks displayed different levels of difficulty, which stably differentiated between the varying degrees of the 102 examinees' pragmatic abilities. The raters showed internal consistency despite their different degrees of severity. Stable fit statistics and distinct difficulties were reported within each of the interaction-sensitive rating criteria, indicating that they contribute to measuring L2 pragmatic competence. In particular, the two rating categories for interactional competence were distinct in their difficulty levels, which supports the CA findings.
Based on the qualitative and quantitative analyses, all of the validity evidence was woven into the validity argument for task-based assessment of L2 pragmatics in interaction, focusing on the four types of inference, domain description, evaluation, generalization, and extrapolation. The current study exemplifies how the construct of L2 pragmatics in interaction can be operationalized based on the qualitative scrutiny of the target performance that is also supported by the quantitative findings, which contributes to the advancement of measuring L2 pragmatics. Lastly, this study provides additional grounds for the recent development of the validity argument approach in the field of language assessment at large.
Description: Ph.D. University of Hawaii at Manoa 2013.; Includes bibliographical references.Thu, 01 Aug 2013 00:00:00 GMThttp://hdl.handle.net/10125/1006562013-08-01T00:00:00ZYoun, Soo JungMotivation and persistence of learning among L2 learners in self-instructionhttp://hdl.handle.net/10125/102006
Abstract: This study examined the relationship between learners' motivation to learn a second language (L2) and persistence in their learning using self-instructional radio (SIR) materials with a sample of Japanese high school students learning English. L2 self-instruction remains under-researched, in spite of the importance of out-of-class learning emphasized in the literature. One example of L2 self-instruction which is prevalent in Japan is learning using SIR materials, with the country's public broadcasting company offering SIR English-education materials since 1934.
One problem related to L2 self-instruction is the high learner drop-out rates and the issue of motivation. Thus, in order to examine self-instructed learners' motivational bases, data collection consisted of a questionnaire and two sets of interviews which were conducted half a year apart. The data were analyzed within the framework of (a) the structure of learners' motivation within the self-determination theory (Deci & Ryan, 1985), international posture (Yashima, 2009), L2 communication confidence, and L2 willingness to communicate (MacIntyre, 2007), and (b) differences in these variables between persistent and non-persistent learners.
The results indicated that intrinsic motivation and identified regulation were important variables related to persistence, as well as the amount of effort made in learning using SIR materials and the quality of learning experiences with these materials. Furthermore, simply persisting in learning using SIR materials did not make learners confident in L2 communication. Practice in L2 communication seemed necessary to be confident in L2 communication, which SIR materials did not offer. Although opportunities for L2 communication were not frequent among the participants, they found innovative ways for L2 communication, such as communicating in English with their siblings. When they were confident in L2 communication, they were likely to be willing to communicate in the L2.
These results suggest that in order to help learners persist SIR materials can have sections (a) stimulating the intrinsic aspect of L2 learning and (b) showing various role models of L2 users and demonstrating the importance of L2 learning. Furthermore, learners may be able to persist by finding their own ways of engaging in L2 communication outside their studies with SIR materials, supplementing the deficiency of self-instruction.
Description: Ph.D. University of Hawaii at Manoa 2013.; Includes bibliographical references.Wed, 01 May 2013 00:00:00 GMThttp://hdl.handle.net/10125/1020062013-05-01T00:00:00ZTakahashi, ChikaRater expertise in a second language speaking assessment : the influence of training and experiencehttp://hdl.handle.net/10125/100897
Abstract: Speaking performance tests typically employ raters to produce scores; accordingly, variability in raters' scoring decisions has important consequences for test reliability and validity. One such source of variability is the rater's level of expertise in scoring. Therefore, it is important to understand how raters' performance is influenced by training and experience, as well as the features that distinguish more proficient raters from their less proficient counterparts. This dissertation examined the nature of rater expertise within a speaking test, and how training and increasing experience influenced raters' scoring patterns, cognition, and behavior.
Experienced teachers of English (N=20) scored recorded examinee responses from the TOEFL iBT speaking test prior to training and in three sessions following training (100 responses for each session). For an additional 20 responses, raters verbally reported (via stimulated recall) what they were thinking as they listened to the examinee response and made a scoring decision, with the resulting data coded for language features mentioned. Scores were analyzed using many-facet Rasch analysis, with scoring phenomena including consistency, severity, and use of the rating scale compared across dates. Various aspects of raters' interaction with the scoring instrument were also recorded to determine if certain behaviors, such as the time taken to reach a scoring decision, were associated with the reliability and accuracy of scores.
Prior to training, rater severity and internal consistency (measured via Rasch analysis) were already of a standard typical for operational language performance tests, but training resulted in increased inter-rater correlation and agreement and improved correlation and agreement with established reference scores, although little change was seen in rater severity. Additional experience gained after training appeared to have little effect on rater scoring patterns, although agreement with reference scores continued to increase. More proficient raters reviewed benchmark responses more often and took longer to make scoring decisions, suggesting that rater behavior while scoring may influence the accuracy and reliability of scores. On the other hand, no obvious relationship was seen between raters' comments and their scoring patterns, with considerable individual variation seen in the frequency with which raters mentioned various language features.
Description: Ph.D. University of Hawaii at Manoa 2012.; Includes bibliographical references.Sat, 01 Dec 2012 00:00:00 GMThttp://hdl.handle.net/10125/1008972012-12-01T00:00:00ZDavis, Lawrence EdwardThe usefulness of accreditation-mandated outcomes assessment in college foreign language educationhttp://hdl.handle.net/10125/100898
Abstract: This dissertation investigated the extent to which accreditation-mandated assessment capacity contributes to assessment use in tertiary foreign language (FL) programs. Four research questions were posed to pursue this concern: What is the nature and extent of assessment capacity in college FL programs? Which assessment uses are occurring in college FL programs? What is the relationship between assessment capacity and assessment use in college FL programs? What is the relationship between accreditation-mandated assessment capacity and assessment use in college FL programs?
Information on assessment activity was collected from 100 FL programs using an online questionnaire. FL educator informants supplied information on how assessment has been used in the program (both assessment findings and processes), as well as information on assessment capacity, including existing institutional assessment support, policies/governance patterns, assessment infrastructures, program-level resources, assessment-related leadership, program ethos, assessment-related collaboration and communication, and additional high-quality assessment-related activities and conditions.
Findings suggest that (a) assessment capacity is highly variable in FL education programs, tending toward being more absent than present, though a number of programs have robust capacity; (b) most programs use assessment for meeting accreditation and institutional requirements, though some use assessment toward increased educational understanding, development, and improvement; (c) the types of assessment capacity most likely to result in improvement-oriented assessment uses are assessment communication and collaboration, program ethos, and other high-quality assessment activities/conditions; and (d) capacity types strongly defined by accreditation assessment mandates (i.e., institutional assessment support and policies/governance patterns) do not have a strong relationship with formative assessment use.
The study makes three main contributions to research knowledge on outcomes assessment washback in college FL programs. First, there is a better understanding of the general assessment capacity elements (an overall assessment capacity framework) needed for productive assessment uses in FL education. Second, the study sheds light on specific capacity elements ostensibly most able to induce assessment use: namely, those elements associated with internally-driven desires for inquiry and change originating within FL programs. Finally, the study shows that institutionally-driven assessment capacity--by itself--is largely unable to create the needed conditions and dispositions for transformative assessment use in FL programs.
Description: Ph.D. University of Hawaii at Manoa 2012.; Includes bibliographical references.Sat, 01 Dec 2012 00:00:00 GMThttp://hdl.handle.net/10125/1008982012-12-01T00:00:00ZDavis, John McEwanLanguage proficiency as a modulator of the processing of unattended texthttp://hdl.handle.net/10125/100561
Abstract: This thesis investigates the modulatory role of proficiency in implicit attentional processes that co-occur with noticing (Schmidt, 1990). The primary hypothesis was that higher Japanese proficiency would go hand in hand with a stronger inhibition of irrelevant stimuli made of Japanese hiragana characters. A secondary purpose was to explore a hypothesized link between saliency and proficiency upon which the primary hypothesis depends.
The findings regarding the inhibitory effect were inconclusive. However, a significant correlation was found between language proficiency, as measured by a lexical decision task, and reaction times in a task originally designed to investigate Inattentional Blindness. This is taken as supporting the hypothesized link between proficiency and the visual saliency of written text. This thesis makes a first step towards bringing methods from Inattentional Blindness research into the field of SLA, and it is hoped that this foundation can be improved and built upon in future research.
Description: M.A. University of Hawaii at Manoa 2014.; Includes bibliographical references.Fri, 01 Aug 2014 00:00:00 GMThttp://hdl.handle.net/10125/1005612014-08-01T00:00:00ZPeters, Ryan ErnestDoing critical language pedagogy in neoliberal spaces : a materialist narrative analysis of teaching young learners of english in a Korean hagwonhttp://hdl.handle.net/10125/101229
Abstract: Critical language pedagogy has been practiced in many contexts, but there have been few reports of critical pedagogy being practiced in neoliberal spaces of private language education. In this thesis, I document critical English language teaching initiatives using the specific case of a South Korean English private language school (hagwon) to demonstrate the possibilities of such an approach in a private institution. Using a critical practitioner research perspective (Cochran-Smith & Lytle, 2009), I collected data from my classes at a hagwon over a 15-month period in the form of artifacts (ballots, student surveys, etc.), images, and student writings. I use emplotment (Polkinghorne, 1995) as a means of creating a narrative from non-narrative data and a materialist analysis (Alaimo & Hekman, 2008) to examine the data. Student resistance, negotiating syllabi, and learner-created materials, and critical episodes in three classes, illustrate the possibilities, need for, and limitations of critical pedagogy in neoliberal spaces.
Description: M.A. University of Hawaii at Manoa 2014.; Includes bibliographical references.Mon, 01 Dec 2014 00:00:00 GMThttp://hdl.handle.net/10125/1012292014-12-01T00:00:00ZWest, Gordon BlaineThe use of English as a local language resource for identity construction in Japanese television variety showshttp://hdl.handle.net/10125/100516
Abstract: This dissertation examines the context of Japanese variety shows to analyze the ways that English is used for identity construction and how this relates to the idea of English as a local resource but still connected to the wider world. It also analyzes the ways that English is used as an entertainment resource for both variety programs and for the people who appear on these shows. Japanese variety shows are a site where tarento, entertainers that are marketed through their personality without belonging to single categories such as singer, actor, or dancer, are often ridiculed and humiliated for the purposes of entertainment. During this process, the tarento will often construct themselves and others through tactics of intersubjectivity that isolate individuals from the larger group using different dimensions of relation. This dissertation takes a Sociocultural Linguistics approach to the data allowing for the combination of multiple methods and traditions to gain a more holistic understanding of the connections between the English Language, Japanese media culture, and society. Drawing upon Bakhtin's (1981) chronotopes, and Bateson's (1972) notion of interpretive frames, I analyze the discourse of Japanese variety shows as a series of embedded frames of different places in space and time that are constructed through indirect indexical reference working together to create a single text. Utilizing data from focus groups and interviews as well as broadcast data this dissertation investigates the content of the text, audience reception and production aspects of Japanese television. I analyze the uses of English to construct images of intelligence and social cool as well as stupidity and unstylishness through indexical fields. The analysis reveals how socially positive and negative images are used as marketable resources and how intermediary level discourse in the form of screen text or telop can iii present itself as having institutional authority through use of typography. My findings reveal that English is used in a continuing cycle connecting the indexical fields of English to the program and the viewership. The findings also show that the educational context of learning English is never totally separated from its use in this medium.
Description: Ph.D. University of Hawaii at Manoa 2014.; Includes bibliographical references.Fri, 01 Aug 2014 00:00:00 GMThttp://hdl.handle.net/10125/1005162014-08-01T00:00:00ZFurukawa, Gavin KenUnveiling linguistic competence by facilitating performancehttp://hdl.handle.net/10125/100350
Abstract: This dissertation looks at the question of whether the source of language learners' poor performance is a deficit in knowledge or a performance problem. Two constructions are examined, both being cases in which learners have been reported to show deficits in linguistic competence: (1) Korean OSV (e.g., Elmo-lul Big Bird-ka anayo. 'Elmo, Big Bird is hugging.'), where knowledge of both scrambling and case markers is necessary for target performance; (2) the English tough construction (English TC; e.g., The dolli is easy to see ei), where knowledge of tough movement is necessary for target performance (i.e., the object interpretation only). If poor performance (operationalized as chance performance or below) is due to a failure to apply knowledge rather than the absence of knowledge, good performance (i.e., significantly-above-chance performance) should emerge with the effective manipulation of non-grammatical factors to facilitate learners' application of knowledge. I experimentally tested this hypothesis in two studies, employing both (a) sentence-internal manipulations and (b) sentence-external manipulations. The Korean OSV study collected data from Korean Heritage children (KHC) (n = 31) as well as two groups of native Korean-speaking children, a younger group (n = 21) and an older group (n = 23). The English TC study collected data from native Korean-speaking adult L2ers of English (n = 49) and native Korean-speaking child L2ers of English (n = 30), along with adult native speaker controls (n = 10).
Study 1: Knowledge of scrambling and case in Korean was assessed, first, via picture-selection comprehension tasks that (a) manipulated the prosodic salience of case markers and (b) manipulated context to make the direct object a natural scrambled topic, and, second, via a production task eliciting case markers. The results suggested three possible sources of poor performance. Many participants showed good performance only in the manipulation conditions (vs. the baseline condition), suggesting that their poor performance in the baseline condition could be due to a performance problem (e.g., perception failure or a heavy processing load). Some participants showed absence or errors of case in the production task, which suggests that their poor performance could come from a deficit in knowledge of case or from a mapping problem between a Case feature and case morphology.
Study 2: Knowledge of the English TC was assessed via Truth-Value Judgment Tasks (TVJTs) that (a) manipulated verb transitivity to make the infinitival object gap more salient or less salient and (b) manipulated context to avoid or strengthen bias toward the (erroneous) subject interpretation. A Korean TVJT was also conducted to find the interpretation tendency for the Korean translation equivalents of the English TC. Comparisons between the Korean and the English TVJT results revealed a strong subject-interpretation bias in both languages for lower proficiency adult and child L2ers but only in Korean for higher proficiency adult L2ers, which suggests that L2ers do transfer their L1 subject interpretation but can subsequently unlearn it----despite an (ostensible) absence of negative evidence. However, (most of) the high proficiency adult L2ers still showed (below) chance performance in the error-inducing conditions, which suggests that non-grammatical factors can conceal knowledge of the English TC that some of them actually have.
The two studies show that poor performance on the part of language learners----here, Heritage learners and L2 learners----is not necessarily due to a deficit in knowledge, which in turn suggests that linguistic competence can be obscured due to performance reasons.
Description: Ph.D. University of Hawaii at Manoa 2014.; Includes bibliographical references.Thu, 01 May 2014 00:00:00 GMThttp://hdl.handle.net/10125/1003502014-05-01T00:00:00ZKim, KitaekThe relative contribution of input modification, learner awareness, and individual differences to second language construction learninghttp://hdl.handle.net/10125/100336
Abstract: This study adopts a constructionist approach to investigating three recent proposals grounded in cognitive perspectives on second language acquisition, which broaden the research agenda in the areas of input modification, learner awareness, and individual differences. Specifically, it was hypothesized that the following would support learning of artificial morphological constructions: (a) input modification in the form of partially repeated sequences, (b) awareness of precise form-meaning mappings, and (c) cognitive-affective learner differences. During the computerized training phase of the study, 90 English-dominant undergraduate participants were placed into three conditions: two experimental conditions exhibiting partial repetition similar to that found in caregiver discourse and a control condition exhibiting no such repetition. The testing phase utilized a picture-word matching task to assess participants' ability to judge correct versus incorrect constructions based on trained and untrained subtest items. Learner awareness was gauged through source attributions and a retrospective written questionnaire. Measures of learner differences included working memory tasks (reading span and dual 3-back) and personality scales (openness and intellect). A test-retest study with 20 of the participants showed these individual difference measures to be relatively stable. Results indicated that the study's hypotheses were largely supported. Learner awareness levels, reading span scores, and item type significantly predicted outcomes on the trained and untrained subtests of the picture-word matching task, based on a regression model including the random effects of items and participants. Also, a better fit to the data was achieved by using a fine-grained, rather than course-grained, system of coding awareness. However, contrary to hypothesis, participants in the experimental conditions did not significantly outperform those in the control condition on the picture-word matching task. Another finding of this study was that participants reported use of both explicit (rule, memory) and implicit (guess, intuition) knowledge sources on the picture-word matching task. In conclusion, this study offers additional evidence concerning the roles of awareness and individual differences in second language learning, suggests ways to enhance the internal validity of research in this domain, and potentially informs future research on input modification as a pedagogic technique.
Description: Ph.D. University of Hawaii at Manoa 2014.; Includes bibliographical references.Thu, 01 May 2014 00:00:00 GMThttp://hdl.handle.net/10125/1003362014-05-01T00:00:00ZJackson, Daniel OwenOn the L2 acquisition of Korean wh-constructions with negative polarity items : adult L2, child L2, and child L1 developmenthttp://hdl.handle.net/10125/20889
Abstract: This thesis investigates the acquisition of Korean wh-constructions with negative polarity items (NPls) in two ways: (i) demonstrating L2 poverty-of-the-stimulus effects (e.g., Schwartz & Sprouse, 2000) and (ii) comparing three different groups of language learners - adult L2ers, child L2ers, and first language (Ll) children - in their developmental sequences (e.g., Schwartz, 1992, 2003). The phenomena investigated are Korean wh-constructions with NPIs. While scrambling in Korean (an SOY wh-in-situ language) is generally optional, in the context of negative questions with an NPI (e.g., amwuto 'anyone'), (i) scrambling of object wh-phrases is obligatory (i.e., OSV) on the wh-question reading (an Intervention Effect, Beck & Kim, 1997) and (ii) the non-scrambled question (SOV) has a yes/no-question reading (exclusively). These properties of Korean wh-constructions with NPIs constitute poverty-of-the-stimulus problems for English-speaking learners as well as for native Korean children. Adult L2ers and child L2ers, independently assessed for Korean proficiency, as well as native Korean children and adults participated in an elicited-production task, an acceptability -judgment task, and an interpretation-verification task. The results show: (i) that adult and child L2ers performed like the native adult controls on all three tasks, indicating that they overcame L2 poverty-of-the-stimulus problems, and (ii) that adult L2ers and child L2ers go through the same development sequences in their acquisition of Korean wh-constructions with NPls. On the basis of these results, it can be argued that UG constrains (adult) L2 acquisition.
Description: Thesis (M.A.)--University of Hawaii at Manoa, 2008.; Includes bibliographical references (leaves 116-119).; xiii, 119 leaves, bound 29 cmTue, 01 Jan 2008 00:00:00 GMThttp://hdl.handle.net/10125/208892008-01-01T00:00:00ZSong, Hyang SukRater bias in assessing the pragmatics of KFL learners using facets analysishttp://hdl.handle.net/10125/20888
Abstract: As interest in research on second language pragmatics increases, some pragmatics research has been done on Korean as a foreign language (KFL) learners. This research has focused on pedagogical aspects of Korean pragmatics and interlanguage pragmatics. However, very little research has been done on the pragmatics assessment of KFL learners, in terms of discussing appropriate test types and whether, certain speech acts and test types affect raters' assessment of KFL learners' pragmatics performance. The focus of this study is on investigating (a) whether factors such as test types and speech acts affect raters' assessments of the pragmatics of KFL learners and (b) which test types are most appropriate for assessing pragmatics of KFL learners. For these purposes, this study analyzes three interactions from the test results using the computer program FACETS (Linacre, 1996): the interactions between rater bias and test types, rater bias and speech acts, and examinees and test types. This study uses three different pragmatics tests adapted from Hudson, Detmer, and Brown's (1995) pragmatics prototype tests: Open-written' Discourse Completion Task, Language Lab, and Role-play. Within each of these three test types are three speech acts: refusal, apology, and request. The results of this research indicate that all three raters showed different degrees of severity in their ratings', depending on the type of speech act. Additionally, each examinee showed different degrees of proficiency depending on the test type. I will discuss which test types are most appropriate for assessing KFL learners' pragmatics performance, how certain speech acts and test types affect raters' assessments, and how what these research findings mean for KFL classrooms.
Description: Thesis (M.A.)--University of Hawaii at Manoa, 2007.; Includes bibliographical references (leaves 71-76).; ix, 96 leaves, bound 29 cmMon, 01 Jan 2007 00:00:00 GMThttp://hdl.handle.net/10125/208882007-01-01T00:00:00ZYoun, Soo Jung