EURODL Mailinglist

EURODL Visitors

Who are the Open Learners? A Comparative Study Profiling
non-Formal Users of Open Educational Resources

Robert Farrow, Beatriz de los Arcos, Rebecca Pitt, Martin
Weller,
Institute of Educational Technology, The Open University, United
Kingdom

Abstract

Open educational resources (OER) have been identified as
having the potential to extend opportunities for learning to non-formal
learners. However, little research has been conducted into the impact of OER on
non-formal learners. This paper presents the results of a systematic survey of
more than 3,000 users of open educational resources (OER). Data was collected
between 2013 and 2014 on the demographics, attitudes and behaviours of users of
three repositories. Questions included a particular focus on the behaviours of
non-formal learners and the relationship between formal and non-formal study. Frequency
analysis shows that there are marked differences in patterns of use, user
profiles, attitudes towards OER, types of materials used and popularity of different
subjects. The experience of using OER is fairly consistent across platforms in
terms of satisfaction and impact on future behaviour. On the whole, non-formal
learners surveyed were highly positive about their use of OER and believe they
will continue to use them. With regards to this making formal study more likely
some degree of polarization was observed: some believed formal study was now
more likely, while others felt it made this less likely. On the whole, while
non-formal learners are enthusiastic about using free and online resources, the
language and concept of OER does not seem to be well understood in the groups
surveyed. A range of findings relating to OER selection and use as well as
differences between repositories are explored in the discussion.

Introduction

According to the definition provided by Hewlett (n.d.),
open educational resources are “teaching, learning, and research resources that
reside in the public domain or have been released under an intellectual
property license that permits their free use and re-purposing by others”. These
can include whole courses of open content, textbooks, multimedia, software and
any other materials which may be used to teach or support learning such as
lesson plans and curricula. There remains some debate about what should qualify
as an ‘open’ resource, with some definitions emphasizing open access to resources
and others focusing on the affordances for revising and repurposing afforded by
open licenses (see Creative Commons, 2013). However, any disagreements tend to
be limited to the specific kinds of licenses for educational that are termed
‘open’ and whether they should permit specific forms of re-use (such as only
allowing non-commercial re-use).

Evaluating the effects that OER have on learning is
problematic for several reasons. Once materials are published as OER they can
be adapted and repurposed in accordance with their licensing restrictions. This
is often done to improve or update a resource, or to make it more closely
aligned to a specific educational need. But this possibility for adaptation can
also make it hard to judge the efficacy of a particular resource, and makes
longitudinal studies problematic. Furthermore, even when data can be collected
comparing similar cohorts who have consistently used OER and non-OER it can be
difficult to reliably attribute any change in performance to the openness of
the resources used.

Most research exploring the impact of open educational
resources (OER) has focused on learners who are registered for a course of
study at an educational institution where OER – often in the form of open
textbooks – are used. A recent review of efficacy and perception studies
(Hilton, 2014) found only twelve instances of peer-reviewed empirical research
where the focus was on OER used as primary learning materials. These studies
have indicated a high level of student satisfaction with the quality of OER
available to them. Students have also consistently reported – particularly in
the case of open textbooks – that OER have helped them financially (Pitt,
2015). OER use has also been correlated with higher test scores and lower rates
of attrition (Hilton & Laman, 2012) though it is more common to find that
use of OER delivers equivalent student satisfaction and performance at
massively reduced cost (Feldstein et al., 2012; Wiley et al.,
2012).

The use of open educational resources (OER) by learners
who are not registered for a formal programme of study has been subject to
increased attention in recent years, with a recognition that there is a need
for greater understanding of how to leverage non-formal learning in support of
formal learning and support the transition from the former to the latter
(Latchem, 2014; McGreal et al., 2014).

Sangrà and Wheeler (2013) have suggested that
non-formal learning should increasingly be viewed as a viable alternative to
formal education, but recognize that more research is needed to establish
effective forms of support. Miyazoe and Anderson (2013) argue that:

“the availability of ever-growing amounts of OER and the
consequent non-formal learning opportunities fuel this ‘opening’ of the
traditional education systems. These free and open opportunities for both
interpersonal and student-content interaction create an interaction surplus
that can be used to augment and enhance formal educational curricula and
systems”.

The emergence of non-formal learning networks, facilitated
by social networking and MOOC platforms, are increasingly recognised as a
possible catalyst for non-formal learning. Siemens (2005) and Downes (2007)
have outlined the ways in which such networks can be understood to support
non-formal learning through the prism of Connectivism, which diminishes the
distinction between formal and non-formal learning.

“Non-formal learning is a significant aspect of our learning
experience. Formal education no longer comprises the majority of our learning.
Learning now occurs in a variety of ways – through communities of practice,
personal networks, and through completion of work-related tasks… Learning is a
continual process, lasting for a lifetime. Learning and work related activities
are no longer separate. In many situations, they are the same.” (Siemens, 2005)

The relationship between non-formal learning and formal
study is increasingly an area for interest for higher education providers. Most
research into non-formal learning concentrates on either the relevance of open
materials for educational institutions, or on recognition of non-formal
learning for professional development. As a result, little is known about the
‘non-formal’ OER learners; their ways of using these resources; or their
reasons for studying in this way.

Miyazoe and Anderson (2013) call for more attention to be
paid to the “changing role of formal education in an era of learning
opportunity where online educational resources and opportunities are readily
accessible and in many cases completely free of cost to the learner”. Advocacy
for non-formal learning tends to be grounded in the role that open resources
can play in supporting formal learners who are registered for a course of study
at an institution. OER can be used in this way to supplement curriculum, brush
up on an area of study, for research purposes, or to otherwise complement
traditional learning activities (Schmid et al., 2015). Much attention has
been paid to possibilities for recognizing and accrediting the learning that takes
place through self directed study which makes use of OER (Yang, 2015). OER also
comprise a strategy for exposing non-formal learners with access to university
level content; affording them opportunities to become more confident with self
directed study. Institutionally produced OER can thus be seen to have both
pedagogical and business dimensions. Many universities contribute to MOOC on
the strategic assumption that in the long run they will be a source of future
registrants.

Part of the reason that non-formal learning has risen to
greater prominence is a growing consensus that a flexible approach to learning
and assessment can contribute to economic development and a richer
understanding of how people learn in situ. But finding adequate ways to
assess the effectiveness of non-formal learning remains a challenge. Mozilla
Foundation (n.d.) launched the ‘Open Badges’ initiative to provide a framework
for providing recognition for non-formal learning achievements through
non-proprietary digital badges, which once awarded can be presented as part of
a portfolio of learning. Badges tend to focus on skill acquisition rather than
retention of ‘academic’ knowledge (Goligoski, 2012). As Glover and Latif (2013,
p.1398) note, badges have the potential to support both student retention
(through increased motivation) and future employability (through a digital
record of achievement, or a portfolio of badges which can be shared with others
as evidence of learning).

A more traditional approach to the recognition of prior or
non-formal learning is to follow private study with an examination equivalent
to those taken by formal students. Conrad et al. (2013, p.46) have argued
that digital and openly licensed learning materials are well suited to form the
basis of a course of private study that can run parallel to institutional
presentation and yet “[t]he greatest barriers to participation in open
assessment and accreditation practices are identified as the lack of
availability of committed staff members to support such activities, and the
potential costs of redeveloping courses as OER”. Institutions who wish to
promote recognition of prior learning therefore need to adopt a coherent,
holistic strategy.

“Within institutions, key factors for the success of open
assessment and accreditation implementation appear to be a reliance on a strong
base of support within the institution—both in terms of leadership and
resources—and an existing culture of openness that includes policies and
practices around the creation and use of OER […] Policies that enable either
open access or recognition of prior learning via credit transfer or RPL are
also important.” (McGreal et al., 2014, p.130)

The advantage of harnessing this kind of non-formal
learning could be significant. Cofer (2000) estimates that every hour spent in
formal study inspires up to four hours of non-formal learning, but insists that
non-formal learning should not be seen as subordinate to formal study but
valuable in its own right. Similarly, Latchem (2014) argues that as much as
70-90% of lifelong learning results from non-formal learning and skill
acquisition; and yet the majority of research focuses on institutional
learning.

Evidently, there remains much to be known about non-formal
learning and how best to integrate it with existing systems of assessment and
accreditation. This study aims to improve our understanding of this topography
by providing information on those who use online repositories of OER for
private study. It should be noted that our aim is not to identify or profile
distinct learning styles (Kolb, 1984) for non-formal learners. Not only has
this approach been criticized as lacking in empirical support (Coffield et al.,
2004) but such a categorization would not be possible without prior work on how
resources are used by non-formal learners; their learning objectives; and study
techniques used.

Methodology

This research took place within the context of OER
Research Hub project which ran from 2013-2015 (OERRH, 2013). OERRH provided a
focus for research on OER, and was designed to describe how openness is making
a difference to learning and teaching practices. The project operated an open
collaboration model and worked with a range of projects, initiatives and
organisations across four education sectors (K12, college, higher education and non-formal learning). In addition to the survey based research presented
here, the project collected data through interviews, focus groups,
institutional visits and through research fellowship. The open collaboration
model involved negotiating the parameters and design of the research according
to the needs of the collaboration partner. In order to ensure that the areas of
focus remained consistent throughout the project the research was guided by
eleven key hypotheses about open education (de los Arcos et al.,
2014). The main hypotheses investigated across the project focus on the impact
of OER on learners and the ways in which open licensing affects sharing and use
of resources:

Use of OER leads to improvement in student performance and
satisfaction;

The open aspect of OER creates different usage and adoption
patterns than other online resources.

In addition, several hypotheses pertained directly to
non-formal learning:

Non-formal learners use a variety of indicators when selecting
OER;

Non-formal learners adopt a variety of techniques to compensate
for the lack of formal support, which can be supported in open courses;

Open education acts as a bridge to formal education, and is
complementary, not competitive, with it;

Non-formal means of assessment are motivators to learning with
OER.

The survey was designed by the research team to collect
evidence relevant to the project hypotheses concerned with non-formal learning
from repository users.

Three OER repositories were selected for the study: iTunes U
(Apple, 2014), OpenLearn (2015) and Saylor Academy (2015). iTunes U makes
use of Apple’s iTunes content delivery systems – primarily used for commercial
purposes – to make educational content available for free. This often comes in
the form of podcasts and videos, and includes OER from museums and other
cultural institutions as well as from higher education. The Open University
(UK) runs has the most popular educational channel on iTunes U, though
most major OER providers are represented in some way. Institutions can set up
their own channel although can’t necessarily upload content to Apple servers. There
is a course builder facility which is designed to help users create a programme
of study within the platform. iTunes U is restricted to users of
proprietary software/hardware made by Apple. Simply put, learners who wish to
access these materials must at some point buy into the Apple ecosystem – even
if only minimally. The advantage of this for the student is the assurance that
all the technology should work adequately together, though there is a case for
arguing that this is a significant impediment to openness and accessibility.

OpenLearn (2014) is The Open University (UK) collection of
free online learning materials, many of which are taken directly from courses
presented to fee-paying students at the University. These include games and
interactive media, materials from both undergraduate and postgraduate
programmes as well as complete certified short courses. More than 33 million
unique visitors have visited the OpenLearn website (which receives about 3
million yearly visitors) (Perryman, Law & Law, 2015).

OpenLearn does more to accommodate non-formal learners who
may lack confidence, providing short introductions to subjects as well as
“interactive games, videos, blogs, [and] podcasts” (OpenLearn, 2015). A
newsletter can also be subscribed to in order to get information about new
courses as they come online. Most of the content on OpenLearn is designed to be
explored through a web browser, and includes some multimedia content. It is
structured so as to encourage the site user to browse their way straight into
the content that interests them rather than ‘choosing a course’ from the onset.
This ‘magazine’ style may be intended to ameliorate the potentially
intimidating nature of the content for learners who are inexperienced or
lacking confidence. On the homepage for OpenLearn there are often links made
between current events (e.g. local and European elections) and relevant course
content. This seems to be designed to make best use of pre-existing user
interests, current TV shows and topicality to encourage the personal inquiry of
the learner as the start of a learning pathway.

The third repository in this study is Saylor Academy. Founded
in 2008, its mission is to use technology to drive the cost of a college
education to zero. Working towards this through curating and compiling open
content, Saylor acts as an aggregator of existing content, compiling OER into
complete courses that emulate or complement existing college courses. Where
Saylor is unable to source adequate open content they commission academic
consultants to write additional content to fill gaps in the curriculum. In
addition to curating college- length courses Saylor also offer a range of featured
pathways through several OR collections. These can offer coverage similar
to an introductory level liberal arts course, or training in core professional
competencies.

Upon completing a course of study with Saylor students can
be examined. For successfully completing a Saylor commissioned programme of
study students can earn a ‘Saylor Certificate of Completion’ which, while
conveying no institutional credit, provides evidence of learning and may be
recognised differently in the future. Hilton, Murphy and Ritter (2014) have
provided an account of the pedagogical and organizational theories informing
the provision of OER as well as describing how these have been applied in the
case of Saylor Academy resources. They argue for improved institutional
recognition of non-formal learning, noting anecdotal evidence that students
have gained college credit after studying non-formally using Saylor materials
and subsequently passing the same invigilated exam as formal learners following
a non-open curriculum.

One thing to note about the different repositories of OER
is that they tend to be characterized by being free at the point of delivery to
learners but the different platforms exhibit different potentials for re-use,
re-appropriation and modularization of learning materials. Though Weller (2014)
has argued that we need to remain open about openness and the
possibility that it can mean different things in different contexts, what
qualifies as open in different contexts remains contested. For some, an open
licence which legitimizes re-use and re-appropriation is a minimal requirement.
In addition, we might expect that materials are natively formatted and licensed
in such a way as to encourage re-use and re-purposing (iTunes U, for
example, would not meet this requirement).

Collecting data from non-formal learners presents
particular challenges around sampling and validity because of difficulties with
verification. A multi-dimensional analysis (including factor analysis and
smallest space analysis) of research literature published in the fields of
non-formal and informal learning concluded that “the familiar qualitative and
quantitative tools already in the hands of researchers, such as interviewing,
observing, surveying, etc. are valuable and appropriate for studying non-formal
education” and that no new tools or approaches were required to produce
meaningful results” (Cohen, 2007).

Data Collection

OER repositories who acted as OER Research Hub
collaborators – iTunes U, OpenLearn, and Saylor – circulated an invitation
to complete the survey to non-formal learners who used these sites between 2013
and 2014. 2299 usable responses were received from users of Saylor Academy, 725
from OpenLearn users and 110 from users of iTunes U. The total sample size
was 3127 (though not all respondents answered all questions and some questions
were omitted from the iTunes U survey). Precise sample sizes for each
question are provided in the reporting below. Data was collected through
SurveyMonkey, refined and compiled before being analysed in SPSS and Excel.

Responses

The survey was answered by OER users from a wide range of
countries, as shown in Figure 1. Most responses came from the USA (n = 862)
or the UK (n = 473) though India (n = 117) Canada (n = 87)
and Brazil (n = 84) also contributed significant amounts of data. Most
countries were represented and people from every continent contributed
information. Excluding Africa, only Bolivia, French Guyana, Greenland,
Kyrgyzstan, Suriname, Turkmenistan, Uzbekistan, and Venezuela were not
represented. Most countries recorded between 5 and 40 responses.

Figure 1. Geographical spread of survey responses

Results

A set of demographic questions across the surveys asked
about age, gender, prior qualifications and employment status to facilitate
profiling OER users. As Figure 2 shows, the repositories exhibit clear
differences in the age profile of their users.

Figure 2. Age profiles of repository users (N = 3127)

Users of iTunes U showed a much lower average age
profile with 71.8% of their users aged below 35. By contrast, OpenLearn users
tended to be older, with 69% aged 35 or over and relatively few younger users. The
pattern of user age profiles was closer to a standard deviation for Saylor
users (perhaps reflective of a larger sample size). Similar patterns of
difference can be discerned when considering the gender of respondents.

Figure 3. Gender profiles of repository users (N=3090)

Figure 3 indicates that Saylor users were more evenly
split between male and female (and had a greater proportion of users who
identified as transgender) while iTunes U users were more slightly more
likely to be male. OpenLearn showed the largest difference in the gender of
their users, with female users outnumbering male users by approximately half.

Non-formal learners were also asked about their highest
academic qualification and their employment status to ascertain differences in
patterns of users across repositories. As is perhaps to be expected given the
age profiles associated with the different repositories (Figure 3) iTunes U
users, generally younger, were most likely to report holding a school leaving
qualification as their highest. But as Figure 3 shows, the general trend
was that users of these platforms tended to report already holding a degree. 59%
of Saylor users held at least an undergraduate degree. The proportion of
university graduates for OpenLearn and iTunes U were 46% and 38.2%
respectively.

Figure 4. Educational profiles of repository users (N = 3038)

The age of non-formal users of OER also seems important
when profiling employment status. Figure 5 illustrates the main differences in
pattern. Saylor users were much more likely to be in full time employment (52%,
n = 2300) while the majority of iTunes U users (43%, n = 110)
were in full time education. iTunes U also had the highest proportion of
users who volunteer part time (15%, n = 110). Approximately 40% of
OpenLearn users were in full-time employment, and OpenLearn had the highest
proportion of retired users (14%, n = 732).

Figure 5. Employment profile of repository users (N = 3142)

OER repository users were also asked about the ways that they
have typically connected to the internet in the previous three months. Figure 6 summarizes this information.

Figure 6. Internet access profiles of repository users
(N = 3116)

When taken in conjunction with the geographical diversity
of the sample these patterns suggest that those accessing these online
resources in developing countries are more likely to do so through a
smartphone, tablet, or home broadband. A small proportion (less than 10%) of
respondents accessed the internet through a dialup connection. iTunes U
users were more likely to access materials through a tablet or while at an
educational institution or community facility.

The final part of background information collected
concerned disability. For the combined sample of all three repositories, 10% of
users disclosed a disability (n = 3160). OpenLearn had the highest
proportion of users who identified as disabled (15.7%. n = 737) while
Saylor reported the lowest proportion of disabled users (7.9%, n = 2298).
The relatively high rate of declaration by OpenLearn students may reflect
similarities to the approach taken to improving accessibility at The Open
University through an inclusive framework which covers human and technical
elements from course production through to evaluation (McAndrew, Farrow &
Cooper, 2012). Figure 7 provides more information on the types of
disability reported, and shows that long term or chronic illnesses and mobility
impairments were more commonly reported by OpenLearn users while mental health
problems more likely to be reported by iTunes U users. iTunes U users
were also much more likely to declare a speech disability. It should be noted
that some respondents declared more than one disability.

Figure 7. Disability profiles of repository users (N = 3137)

With respect to reasons for using OER outside of a formal
educational context, by far the most common response was a personal interest in
a subject. Professional development and study relating to work were also
popular. Particularly with users of Saylor. OpenLearn and Saylor users were
more likely to report using OER to support the development of study skills or a
second language. iTunes users were more likely than others to seek out OER for
the purpose of sharing with others. Generally, iTunes users reported fewer
reasons for accessing OER. Responses to this question are summarized in Figure 8.

Figure 8. Reasons for using OER (N=2783)

The next part of the survey asked about the subject areas
where open resources are typically used by the cohort. The responses are shown
in Figure 9, where the most striking pattern is that users of iTunes U
reported using OER across a much wider range of subject areas than users of the
other repositories. This could be taken to suggest that users of this platform
are more inquisitive and diverse in their approach to self-study, or the fact
that older casual learners access a broader range of OER as they indulge their
curiosity about several subjects. Alternatively, this trend may reflect
elements of the podcast model which allows users to download several resources
that they are interested in at once to listen to later. In general, there was
quite an even spread of interest across all subject areas.

Figure 9. OER repository use by subject area (N=2356)

Across the entire sample (n = 2356) the most
popular subjects for OER were computing (31.1%, n = 732) and
economics (30.4%, n = 716). Table 1 shows the most popular
subject areas by platform.

Table 1: Most popular subject areas across repositories
(N = 2356)

Respondents were also asked about the type and format of
the resources they access. Videos, ebooks and open textbooks were the most
commonly used resources. On the Saylor Academy platform ebooks and open
textbooks were more popular than video. Saylor users reported using less
multimedia content, games and quizzes but more lectures. No non-formal learners
in any group reported using open data sets or lesson plans in their studies. Figure 10
summarizes the responses to this question.

Figure 10. Types of OER used (N = 2085)

Users of Saylor and OpenLearn were also asked about their
own reasons for accessing OER. (This question was not part of the iTunes U
survey.) Figure 11 summarizes their responses. The most popular reason
across both platforms was the chance to study at no cost. Saylor users were
marginally more likely to cite the flexible and online nature of the resources
while OpenLearn users were more likely to state that the chance to try
university level content was important to them.

Figure 11. Reasons for using OER (N = 3025)

These learners were also asked whether they did more than
simply consume OER as it is presented to them through the platform they most
often use; i.e. did they involve themselves in the kinds of re-use behaviours
that are encouraged and legitimized by open licences? A relatively small number
elected to answer this question, which may indicate a lack of confidence with
the terminology used. Figure 12 presents their responses. It shows that
Saylor users were much more likely to report engaging in processes that support
OER production and evaluation. Although the sample size for this question was
smaller, it is remarkable that more than 80% of Saylor users who answered claim
to have adapted open materials for their own purposes as non-formal learners. (iTunes U
users were not asked this question.) This is an interesting and
counterintuitive result, possibly affected by different understandings around
the concept of adaptation.

Figure 12. Behaviours relating to use/re-use/review of
OER (N = 907)

Survey respondents were also asked about the other
repositories of OER that they have used to assess the extent to which platforms
are used exclusively or in conjunction with each other. The results show that
YouTube is the most popular place to find open resources, with over 50% of each
sample reporting that they used it to find OER. iTunes U and TED talks
were also popular across the samples, as was Khan Academy (though less so with
users of OpenLearn).

Saylor users were much more likely than the other groups
to be studying via MOOC platforms (41.7%, n = 751). Figure 13
presents patterns in OER repository use.

Figure 13. Patterns of OER repository use (N = 2460)

To further understand patterns of use, the survey also
asked about the indicators that non-formal learners look for when selecting OER
(Figure 14). (Note that iTunes U users were not asked this question.)

We see a striking similarity between the responses given
across the two samples. The most important criterion (for approximately 70%)
was relevance to a particular need; i.e. OER were sought to plug a specific gap
in knowledge or skills. The reputation of the OER creator (whether an
individual or repository), clarity around learning outcomes or objectives, and
ease of download or access were all cited as important. Less important were
reviews of OER or personal recommendations. Open or Creative Commons licensing
allowing adaptation was only thought important by a minority of 12-16% which is
interesting given the high level of adaptation rates reported elsewhere (see
Figure 12). It may be that non-formal learners adapt OER without paying
much attention to licensing permissions.

The least important factors appear to be the
attractiveness of the presentation of a resource and whether an OER has
previously been used in an institutional context. In the case of the latter it
should be noted that despite this, more than 50% of the samples thought that
the reputation of the resource creator was important and this reputation is
often closely associated with institutional affiliation. Both Saylor (20%) and
OpenLearn (21%) users identified a resource being required for formal study as
important, suggesting that some of these learners may be using OER in lieu of
non-open materials required for their formal studies.

Figure 15. Challenges faced when using OER (N = 1669)

It is also noteworthy that between 16% and 25% of each
sample believed that lack of support from a teacher or tutor was a challenge to
their learning with OER. This could be seen to reflect a general level of
confidence around independent, non-formal study among these groups, but still
identifies a significant proportion who feel that they would benefit from more
support of this kind.

OpenLearn and Saylor users were asked in more detail about
the techniques they employ to structure their learning when they have no formal
support. Figure 16 shows their responses.

Figure 16. Support techniques used by non-formal
learners (N = 1892)

The general pattern suggests that most non-formal learners
on these platforms are not using many of the techniques described. Saylor users
were more likely to report using a calendar or journal to organize their study
as well as using specialized note-taking software; no OpenLearn user reported
using either of these. Writing study notes was in general more common among
Saylor users (60.2%) while OpenLearn users were more likely to use a blog to
record and organize their learning. Significant proportions of both samples
said that they discussed their learning with others either face-to-face, via
online forums, on social networks, or, less commonly, through micro-blogging
platforms like Twitter or Tumblr.

Finally, the survey asked about the likely impact of OER
use in terms of the future behaviour of the learners. Figures 17-19
summarise their responses (with values for ‘more likely’ behaviours
highlighted).

We can discern here a very similar pattern of satisfaction
with the quality of open materials across the three samples. Between 69% and
74% would recommend the repository to others and approximately half would share
them directly with others (though the figure was slightly lower for OpenLearn
at 44%). High proportions across all samples indicated that they would download
further materials from the repository and feel empowered to undertake further
study in a related area. But this satisfaction with the learning experience may
also translate into a polarization between those who find that non-formal use
of OER has made it more likely that they will seek formal study, and those who
feel that it has made it less likely; presumably because of satisfaction with
the kind of learning materials that are on offer openly. In each case those who
answered ‘more likely’ or ‘less likely’ to this were about equal in number. OpenLearn
users reported slightly greater likelihood of going on to formal study; this is
perhaps reflective of the structured pathways leading from OpenLearn to degree
programme course credits.

The vast majority of those surveyed said that they would
continue to use OER in the future – 84.8% across the three samples as a whole. A
mere 0.4% of those who participated in this study said that using OER had made
it less likely that they would use OER in future.

Discussion

This dataset provides us with a more detailed sense of how
non-formal learners using OpenLearn, iTunes U and Saylor Academy materials
select and use OER. We can distinguish some broad patterns in the user bases of
the different repositories. With a majority of the sample below the age of 30,
iTunes U users were much more likely to be younger and were mostly male. They
are often in full time education and use OER on an informal basis outside of
their formal studies. By contrast, Saylor Academy users are more likely to be
in employment and already in possession of a degree. They tended to be middle
aged and primarily motivated by professional development. OpenLearn users were
more likely to be older, retired, and female, and had a higher proportion of
users who were motivated mainly by personal interest (though 40% are in full
time employment).

Data about prior qualifications supports the pattern often
seen in MOOC research (Laurillard, 2014; Emmanuel, 2013) where users of free
resources are often already graduates of higher education institutions. This
was particularly evident with users of Saylor materials, more than half of whom
(59%) were already in possession of at least an undergraduate degree. This was
not the case for iTunes U, which had a markedly younger user base.

One standout finding is that users of OER across all
platforms expressed a high degree of satisfaction with the OER that they had
accessed. Attitudes toward OER were overwhelmingly positive across the sample
as a whole. This strongly supports the idea that OER (at least for the
repositories studied) were of sufficient quality and appropriate for the
education level of users. However, as Figure 19 indicates, this
satisfaction does not always translate to a willingness to study formally. In
fact, people who have used OER tend to become somewhat polarized. On the one
hand we have those who feel more inspired or more confident after non-formal
use of OER; while on the other there appear to be those whose learning needs
are entirely met by use of open resources. Table 2 summaries this polarization.

The higher figure for OpenLearn may be explained by the
existence of planned pathways between OpenLearn content and degree level
content provided by The Open University (UK) which are intended to facilitate
the transition from non-formal to formal study.

Even as learners reported a high level of satisfaction
with the materials provided by their respective repositories, many identified
difficulty locating subject-specific resources of adequate quality to be a
significant challenge (Figure 15). In light of this it is noteworthy that
there is a certain amount of ‘brand loyalty’ apparent with most learners using
a small number of repositories (Figure 13). This can in part be attributed
to lack of clear understanding of the nature of OER and what qualifies as a
repository. It seems significant that specialist repositories of OER like
Merlot, Jorum, and Curriki were hardly visited at all by the non-formal
learners who answered the survey. This suggests that learners who use free
online digital materials rarely visit specialized OER repositories, perhaps
because they are only aware of resources being provided online for free rather
than the concept and language of openness and OER. Respondents typically
indicated a lack of understanding about the nature of an OER repository: between
9% and 20% of each sample said that they did not use any OER repositories
despite the fact that they had only been offered the chance to participate in
the survey specifically because of their OER repository use.

One standout finding (Figure 12) was the high
reported level of adaptation of OER by non-formal learners (as high as 85% for
Saylor Academy users but still over 20% for OpenLearn). This was not
anticipated and may indicate an unclear question. The research team intended by
adaptation to mean some sort of editing or remix of resources in order to meet
a perceived learning need. Given these responses, it may have been interpreted
as collecting and curating resources together; or as taking material from one
country/level in order to study in a different context. Comments received in
relation to this question were not insightful, but did suggest that most
informal OER users are not confident enough to remix and revise OER: not least
because of the lack of interest in open licensing (Figure 14). This is an
area where follow-up qualitative research focused on illuminating understandings
of ‘adaptation’ would be beneficial.

A lack of knowledge about the technicalities of OER could
be taken to imply that non-formal learners may not have much sense of
reflecting on their own activity. Yet at the same time clarity of learning
objectives was one of the most important factors affecting OER selection, so
learners do seem to care about some aspects of pedagogy. In relation to this,
it is noteworthy that no more than 25% of each sample felt that lack of tutor
support was a barrier to them learning using OER (Figure 15). This
confidence could be reflective of a number of factors, including faith in the
quality and presentation of OER; skills gained through prior study; or
overconfidence in ones ability to study unsupported. Data received about the impact
of OER use on future behaviour would favour the first interpretation. As Figures 17-19 show, most users are overwhelmingly positive about their experience of using
OER across all three repositories.

Limitations and future research

It should be noted that those who responded to the survey
invitations constitute a self-selecting, self-reporting sample. A further
limitation of the data is that respondents were free to skip any question they
did not wish to answer. This means that there are some gaps in the data
provided. Furthermore, in discerning patterns between the different user groups
it should be borne in mind that the sample sizes were quite different across
the three platforms. This may be particularly pertinent for the relatively
small sample size of iTunes U users. Future studies could apply more
rigorous sampling methods in order to extrapolate to more general populations. The
lack of prior research in this area means that even with these caveats the
results are likely to be of interest to a range of stakeholders. Further
analysis of this data set (Farrow et al., 2015) is encouraged. Such work
could examine patterns of response according to country; employment status;
area of study; prior study; and other variables.

Two areas that would likely make a good focus for
qualitative work are to better unpack what non-formal learners understand by
phrases like ‘adaptation’ and ‘relevance’, since these appear to be key
considerations in the selection and use of OER. This may also facilitate
understanding of why some learners ultimately embark on formal study as a
result of their use of online resources.

Conclusion

This paper has presented data collected from users of
three prominent repositories of OER. It has shown that there are marked
differences in patterns of use, user profiles, attitudes
towards OER, types of materials used and popularity of different subjects. Data
also indicate that the experience of using OER is fairly consistent across
platforms in terms of satisfaction and impact on future behaviour. Overwhelmingly,
non-formal learners are very positive about their experiences of using OER,
with a huge majority stating that they are more likely to use OER in the
future. Indeed, such is the level of apparent satisfaction with OER use that it
makes a significant minority less likely to (re-)enter formal education at all.
While users are enthused about using free and online resources, the language
and concept of OER does not seem to be well understood in the groups surveyed.

In relation to the research hypotheses of OER Research
Hub, we can offer the following concluding comments in relation to this study.

Table 3: Summary of findings in relation to OER
Research Hub research hypotheses

Hilton,
J. (2014). A Review of Research on the Perceptions, Influence and
Cost-Savings of OER: Looking Back and Looking Forward. Paper presented at
Open Education 2014. Washington D.C., USA 19th November 2014.

Malcolm,
J., Hodkinson, P., & Colley, H. (2004) Informality and Formality in
Learning: A Summary of the Report to the Learning and Skills Research Centre. London: Learning and Skills Research Centre. Retrieved from
https://kar.kent.ac.uk/4647/

Wiley,
D., Hilton, J. Ellington, S., & Hall, T. (2012). A preliminary examination
of the cost savings and learning impacts of using open textbooks in middle and
high school science classes. International Review of Research in Open and
Distance Learning. 13(3), 261-276.

Acknowledgement

This study was funded by The William and Flora Hewlett
Foundation as part of OER Research Hub. OpenLearn, Saylor Academy and The Open
University iTunes U channel were research collaborators who assisted with
data collection, and the authors gratefully acknowledge their support.