unrestrictedly move from one topic to another without concern forpredetermined order or sequence, (McManus, 2000, p. 221). Conse-

Internet and Higher Education 12 (2009) 16

Contents lists available at ScienceDirect

Internet and HigRunnels, Thomas, Lan, Cooper, Ahern, and Liu (2006) indicated manyimportant issues of online instruction and learning which have yet tobe addressed in research. As such, existing principles and theories ofeducational psychology need to be reexamined or modied to reectunique characteristics of the online learning environment and help usto understand online teaching and learning (Broad, 1999).

Among the different course delivery formats in distance education,the Internet has been the medium of choice for most institutions.Additionally, the Internet is also being used to supplement instruction

Lan, 1996; Orange, 1999). If these self-regulatory learning skills areimportant to the success of learning in the traditional face-to-faceclassroom, it can be expected that these self-regulatory learning skillswill play an even more important role in learning in the onlineenvironment. Students lacking self-regulatory learning skills maymisconstrue the autonomy of the online learning environment and, asa result, may not accomplish learning tasks they are expected in onlinecourses. However, the role of self-regulatory skills in the online learningenvironment has not received the same attention as it does in thethereby changing the traditional face-to-faceOne distinguishing characteristic of online lestudents experience in the learning enviroinstruction eliminates the limitation of plmaterials and to a great degree gives student

was the most benecial for the students with an internal locus ofcontrol who believed they had control over events and situations intheir lives. Bowen (1996) concluded that students with internalaccountability beliefs generally perform better than students with anexternal locus of control in online courses.

As the online learning environment is characterized with autonomy,self-regulation becomes a critical factor for success in online learning.technologies (Lewis, Snow, Farris, & Le 0). According to the U.S. quently, Bowen (1996) found that the online learning environmentcolleges and universities providing courses and degree programs viadistance educationeducation or training courses delivered to off-Measuring self-regulation in online and b

Since the mid-1990s, there has been a boom in the number of U.S.course delivery format.arning is the autonomynment. As such, onlineace, time, and physicals the control over when,

).

rights reserved.nded learning environments

sland Paton b, Shu-Ling Lai c

301, United States

gulated Learning Questionnaire (OSLQ) to address the need for an instrumentonline learning environment, this study provides evidence toward the reliabilityata were collected from two samples of students. The rst sample of studentscourse formatwhile a second sample of students took coursework delivered viat. Cronbach alpha () and conrmatory factor analyseswere performed to assessthe OSLQ across both samples of students. Results indicate the OSLQ is an

what, and how to study (Cunningham & Billingsley, 2003). Research-ers contend that this autonomy gives students the freedom to

her Educationtraditional face-to-face environment.In one study that we found that investigated the relationship

between self-regulated learning and academic performance in onlinecourses, McManus (2000) adapted an Aptitude Treatment Interaction(ATI) paradigm to investigate effects of the aptitude variable of self-regulated learning and two treatment variables of linearity of informa-tion presentation and availability of advanced organizer on educators'learning on applications of computer software. Students' self-regulated

learning was measured by the Motivated Strategies for LearningQuestionnaire (MSLQ; Pintrich, Smith, Garcia, & McKeachie 1993),then, categorized into three levels as low, medium, and high. Instruc-tional materials were presented in six different manners constituted bythree levels of nonlinearity (low, medium, and high) and two levels ofavailability of advanced organizer (presence vs. absence). Students'declarative knowledge was measured by a multiple-choice test, andtheir procedural knowledgewasmeasured by 20 computer applicationsin an authentic situation. McManus found that the advance organizerhelped students when materials were presented with low or mediumlevels of nonlinearity but had a detrimental effect on learning wheninformation was presented with a high level of nonlinearity. Interest-ingly, he foundmixed effects between self-regulation and non-linearity,that is, with the low nonlinearity of presentation, low andmedium self-regulated learners performed better than highly self-regulated learners,with the medium nonlinearity of presentation, the three self-regulated

given the dramatic differences between the two learning environ-ments. As the researcher indicated in his explanation of theinteraction effects between self-regulated learning and nonlinearity,it may be that the online environment does not lend itself to self-regulation strategies use and is unsuitable for these (high andmediumself-regulated) learners, (McManus, 2000, p. 243).

More recently, Lynch and Dembo (2004) examined the relationshipbetween self-regulation and online learning in a blended learningcontext. Lynch and Dembo (2004) found that self-efcacy and verbalability appeared to signicantly relate to academic performance in theform of course grades. In conducting their study of 94 undergraduatesin a blended learning context, Lynch and Dembo (2004), however, didnot utilize a measure of self-regulation contextualized to the online orblended learning environment. In this instance, a measure of self-regulation of learning for the online and blended learning contextswould be particularly useful for researchers as research continues toindicate a positive relationship between these self-regulatory learningskills and academic performance as contextualized to the online orblended learning environments (Chang, 2007).

The purpose of the current study is to examine the psychometricproperties of an instrument created tomeasure a student's ability to self-regulate their learning in environments that arewholly or partiallyweb-based. In creating the Online Self-regulated Learning Questionnaire(OSLQ), we sought to examine its psychometric properties (e.g.reliability and validity) across students experiencing a blended orhybridcourse delivery format as well as a wholly online course format. A

The ndings that students with higher levels of self-regulatedlearning performed signicantly poorer than those with lower level ofself-regulated learning in both low and high nonlinearity presentationconditions could be puzzling for self-regulated learning researchers. Apossible explanation for the puzzling ndings could be the measure-ment used to classify learners into different levels of self-regulation inlearning. With most researchers believing that self-regulation is acontext-specic process (see Zimmerman,1998), an instrument that isvalid in the traditional learning environment, such as the MSLQ(Pintrich et al., 1993), may become invalid in the online environment,Fig. 1. OSLQ path diagram for blenblended or hybrid course format consists of instruction that takes placein both the online and traditional face-to-face learningenvironments. Toachieve this purpose, we examined the instrument across two samplesof students. The rst study sample consisted of students enrolled in ablended course delivery format while the second study sampleconsisted of students enrolled in an online course delivery format.

1. Study 1: method

1.1. Participants

The sample consisted of 434 students enrolled in a course having ablended or hybrid class format at a large, public university located inded course format students.

the Southwestern United States. Of those who self-selected toparticipate, 54% (n=235) of the sample were female while approxi-mately 74% (n=321) of the sample identied themselves as White or

evaluation. The scores obtained from the measure demonstratedadequate internal consistency of scores with =.90. Nunnally (1978)has suggested that score reliability of .70 or better is acceptable whenused in basic social science research such as in this study. Whenexamining the internal consistency of scores by subscale, values forCronbach alpha ranged from .67 to .90 revealing sufcient scorereliability on the subscale level. Table 1 contains internal consistenciesfor scores obtained from each of the subscales for students enrolled inthe blended course format. Appendix A contains a complete copy of theinstrument including subscale construct designation.

1.3. Procedure

The OSLQ was administered online. After data were collected, someitems were recoded and reversed per instrument instructions. Nomodications were made to the instrument. All participants wereassured that their responseswould remain anonymous and condential.Data were imported from the Web into a MS Excel format and then

3L. Barnard et al. / Internet and Higher Education 12 (2009) 16European American followed by 13% (n=56) identifying themselves asHispanic, 9% (n=39) identifying themselves as African American, and4% (n=18) identifying themselves as Asian Americans. A total of 18different academic disciplines were represented and resided approx-imate to the university given the nature of the blended course format.The majority of students enrolled were liberal arts or business majorscompleting the course to satisfy a general education core requirementfor the university studied. The cumulative earned credit hours of thestudent respondents ranged from 0 to 158 h with a mean of 43.03earned hours and a standard deviation of 10.51 h. This distribution ofcumulative earned hours indicates that the majority of students wereclassied as freshman and sophomores. Students were recruited froma pool of 936 students enrolled in a computer-based literacy coursethat satises a university general education requirement. As 434students self-selected to participate in the study from this samplingframe, the resulting response rate was 46%.

1.2. Measure

The Online Self-regulated Learning Questionnaire (OSLQ; Lan,Bremer, Stevens, & Mullen, 2004; Barnard, Paton, & Lan, 2008) is a24-item scale with a 5-point Likert response format with a 5-pointLikert-type response format having values ranging from strongly agree(5) to strongly disagree (1). The OSLQ was developed from an 86-itempool and then examined for their internal consistency and exploratoryfactor analyses results for data collected. Higher scores on this scaleindicate better self-regulation in online learning by students. The OSLQconsists of six subscale constructs including: environment structuring;goal setting; time management; help seeking; task strategies; and self-Fig. 2. Location of online course fimported into SPSS (v. 12.0). Analyses were performed inMPlus (v. 5.10;Muthn & Muthn, 2008). Values for missing data were handled usingfull information maximum-likelihood (FIML) as the method of estima-tion. As anextensionofmaximum likelihood, FIML takes advantage of allpossible data points in analysis. Enders and Bandalos (2001) indicatedthat full information maximum-likelihood is superior to listwise,pairwise, and similar response pattern imputations in handling missingdata that may be considered ignorable.

1.4. Analyses

A higher order, conrmatory factor analyses was performed toestablish evidence towards the construct validity of the measure(Kerlinger, 1986). In performing the conrmatory factor analysis, vestatistics reecting t were reported: the chi-square goodness of tstatistic (2); the ratio of chi-square statistic to degrees of freedom(2/df); the root mean square error of approximation (RMSEA); theTucker Lewis Index (TLI), also known as the Non Normed Fit Index(NNFI); and the Comparative Fit Index (CFI). No post hoc modelmodications were made.

2. Study 1 results

The chi-square goodness-of-t statistic was signicant indicatingthat the model may t the data, 2(246)=758.79, pb .05. The chi-square statistic has been indicated as being sensitive to sample sizethus an adjunct discrepancy-based t indexmay be used as the ratio ofchi-square to degrees of freedom (2/df). A 2/df ratio value less than 5has been suggested as indicating an acceptable t between theormat students by zip code.

hypothesized model and the sample data (MacCallum, Brown, &Sugawara, 1996). With a 2/df ratio value of 3.08, the proposedmodel may have an acceptable t. The root mean square error ofapproximation (RMSEA) as compensating for the effects of modelcomplexity was 0.04, which according to Browne and Cudek (1993)indicates an acceptable t of themodel being less than or close to 0.05.The value of Tucker Lewis Index (TLI), also known as the Non NormedFit Index (NNFI) was .95 and value of the Comparative Fit Index (CFI)was .96. Hu and Bentler (1999) note that t index values of .95 or closeto it are indicative of good t. Thus, the model appears to t the datawell as seen in Fig. 1. The paths in the model were all signicant withstandardized values ranging from .43 to .77. Table 2 contains thestandardized path coefcients from the latent variable constructs tothe items. Results indicate evidence towards the construct validity ofthe OSLQwith respect to students enrolled in a blended course format.

white (n=168). The student gender distribution (73 males versus 131females) in this study is representative of those enrolling in distanceeducation courses across the nation (Kramarae, 2001). The studentethnic/racial distribution in this studywas representative of the studentpopulation of the university studied. A total of 24 different academicdegree programs and a total of 146 different U.S. postal zip codes wererepresented. Fig. 2 contains the locations of all students enrolled in thewholly online course format by zip code. There were approximatelythree students who were located internationally, whose locations arenot represented on the map in Fig. 2.

3.2. Measure

The same measure employed in Study 1was utilized in Study 2. ForStudy 2, the scores obtained from the measure demonstrated anadequate internal consistency of scores with = .92. When examiningthe internal consistency of scores by subscale, values for Cronbachalpha ranged from .87 to .96 revealing sufcient score reliability on thesubscale level. Table 3 contains the internal consistencies for scoresobtained for each subscale for students enrolled in the online courseformat.

The study consisted of a sampling frame of 628 unduplicatedstudents with working (deliverable) e-mail addresses enrolled in onlinecourses at a large, public university located in the Southwestern UnitedStates. Of these students taking online courses, 204 self-selected tocomplete the survey online by responding to a recruitment e-mailmessage resulting in a 32% response rate. Participants were informed asto the voluntary nature of the study. Participants were also assured as tothe condentiality of their responses. Approximately thirty-six percentof the participants identied themselves as male (n=73) and 82.6% asFig. 3. OSLQ path diagram for on3.4. Analyses

The same analyses were performed in Study 1 as in Study 2.

4. Study 2 results

The chi-square goodness-of-t statistic was signicant indicatingthat the model may t the data, 2(246)=680.57, pb .05. The chi-squarestatistic has been indicated as being sensitive to sample size thus anadjunct discrepancy-based t index may be used as the ratio of chi-square to degrees of freedom (2/df). A 2/df ratio value less than 5 hasbeen suggested as indicatinganacceptablet between thehypothesizedmodel and the sample data (MacCallum et al., 1996). With a 2/df ratioline course format students.

achievement as the learningwould be primarily personally managed insuch contexts such as the online and blended learning environments.The OSLQ would be a helpful measure to address this specic query. Asutilized in future research, the OSLQ could be utilized to investigatewhether increases in students' self-regulatory skills in online andblended courses, personally managed, student-centered learning con-texts, are associated with increases in overall academic achievement.

Self-regulated learning is indeed a process that may uctuate andchange with each variation within the learning context. The advantageof using the OSLQ is that it expands self-regulation research into theonline learningdomain. Researchers can employ theOLSQ to assess self-regulatory learningskills aswell as changes in theuse of these skills in anonline learning format and therefore provide key information aboutboth the learner and the context in which the learning takes place.Schraw (2007) has suggested that self-regulatory learning skills can beenhanced by computer-based instruction found in the online andblended learning environments. Findings from a large, multi-national

5L. Barnard et al. / Internet and Higher Education 12 (2009) 16value of 2.77, the proposed model may have an acceptable t. The rootmean square error of approximation (RMSEA) as compensating for theeffects of model complexity was 0.06, which according to Browne andCudek (1993) indicates an acceptable t of the model being less than orclose to 0.05. The value of Tucker Lewis Index (TLI), also known as theNon Normed Fit Index (NNFI) was .93 and value of the Comparative FitIndex (CFI) was .95. Hu and Bentler (1999) note that t index values of.95 or close to it are indicative of good t. Thus, the model appears to tthe datawell as seen in Fig. 3. The paths in themodelwere all signicantwith standardized values ranging from .46 to .84. Table 4 contains thestandardized path coefcients from the latent variable constructs to theitems. Results indicate evidence towards the construct validity of theinstrument for students enrolled in an online course format.

5. Discussion

When having the option, Roblyer (1999) found that students choseonline course formats over traditional face-to-face courses. It is clearthat the enrollment of students delivered by blended and onlinecourse formatswill only increase thus the development and validationof the OSLQ to assess self-regulatory skills in the blended and onlinelearning environments is timely.

The results of both Study 1 and Study 2 indicate evidence towardthe reliability and validity of the OSLQ to assess the self-regulatorylearning skills of students enrolled in both blended and online courseformats. The OSLQ intends to provide a means of assessing the self-regulatory learning skills of students in both the online and blendedlearning environments. In the current study, the two sampleswere notcombined to test for different levels of measurement invariance giventhat the data for the students in the blended and online course formatswere collected at different points of different academic semesters.Future research should consider testing for measurement invarianceby collecting the data for both groups of students at approximately thesame point in a semester. Additionally, those students in the blendedcourse format sample were taking at least one course that had a

Zimmerman (2008) noted that vast amounts of research on learningand performance are evident of the importance of self-regulatedlearning in any learning context whether it may be online, blended, orface-to-face. The development of self-regulated learning is a proactiveprocess and thus as the context of students' learning changes anddevelops so must the methods of measuring self-regulation evolve. Assuch, Zimmerman questionedwhether these new contexts inwhichwemeasure self-regulation could address the question of whether changesin students' use of self-regulatory skills are linked to overall academicstudy identied several aspects of technology-enhanced learningenvironments that had the potential to support self-regulated learning(Steffens, 2006). These aspects include but are not limited to anenvironment that provided more opportunities for interaction, con-tained a feedback and self-monitoring system, used a method of cog-nitive apprenticeship (i.e., coaching), and supported self-efcacy beliefsandoptimistic attributions among learners (Steffens, 2006). Anonlineorblended learning environment that incorporated these aspects maysuggest enhance self-regulated learning skills yet there is no evidenceas of yet to support the claim. To provide this evidence (negating orsupporting), researchers could employ the OSLQ across similar onlinelearning formats and across time to help identify those online andblended learning environments that truly have potential for thefacilitation of self-regulatory learning skills.

The many applications of the OSLQ in future research are as diverseas the many contexts of online and blended learning that take place. Asthe number of students enrolling in online and blended courses in-crease, it is likely that the variation in the methods of teaching andlearning that occur will also increase. Currently, online courses areexpandingbeyond the traditional, face-to-faceeducational environmentwhile becomingmore andmore incorporated into economic institutionsinterested in enhancing the skill level andabilities of itsworkers. For thatreason, self-regulatory processes may come to play a larger and moresignicant role within online and blended course format than it haspreviously in traditional, face-to-face education. Thus, the developmentand further validation of an instrument like the OSLQ becomes relevantand even necessary given the need to assess courses and learners inemerging online and blended learning environments. The OSLQ wouldbe one of manymethods of assessing courses and learners with specicrespect to how these courses facilitate self-regulatory learning skills andhow learners develop these skills.

Appendix A

Item Subscale

1. I set standards for my assignments in online courses. Goal setting2. I set short-term (daily or weekly) goals as well as long-termgoals (monthly or for the semester).3. I keep a high standard for my learning in my online courses.4. I set goals to help me manage studying time for my online courses.5. I don't compromise the quality of my work because it is online.6. I choose the location where I study to avoid too much distraction. Environment

structuring7. I nd a comfortable place to study.8. I know where I can study most efciently for online courses.9. I choose a time with few distractions for studying for my onlinecourses.10. I try to take more thorough notes for my online courses because notesare even more important for learning online than in a regular classroom.

12. I prepare my questions before joining in the chat room and discussion.13. I work extra problems in my online courses in addition to the assignedones to master the course content.14. I allocate extra studying time for my online courses because I knowit is time-demanding.

Timemanagement

15. I try to schedule the same time everyday or every week to study formy online courses, and I observe the schedule.16. Although we don't have to attend daily classes, I still try to distributemy studying time evenly across days.17. I nd someone who is knowledgeable in course content so that Ican consult with him or her when I need help.

Help seeking

18. I share my problems with my classmates online so we know what weare struggling with and how to solve our problems.19. If needed, I try to meet my classmates face-to-face.20. I am persistent in getting help from the instructor through e-mail.21. I summarize my learning in online courses to examine myunderstanding of what I have learned.

Selfevaluation

22. I ask myself a lot of questions about the course material when studyingfor an online course.23. I communicate with my classmates to nd out how I am doing in myonline classes.24. I communicate with my classmates to nd out what I am learning

Roblyer, M. D. (1999). Is choice important in distance learning? A study of studentmotives for taking Internet-based courses at the high school and communitycollege levels. Journal of Research on Computing in Education, 32(1), 157.

Zimmerman, B. J. (2008). Investigating self-regulation and motivation: Historicalbackground, methodological developments, and future prospects. AmericanEducational Research Journal, 45, 166183.that is different from what they are learning.