Although research on the use of Twitter in support of learning and teaching has become an established field of study the role of Twitter in the context of Massive Open Online Courses (MOOCs) has not yet been adequately considered and specifically in the literature. Accordingly, this paper addresses a number of gaps in the scholarly interface between Twitter and MOOCs by undertaking a comprehensive mapping of the current literature. In so doing the paper examines research design through: data collection and analysis techniques; scope and scale of existing studies; and theoretical approaches and underpinnings in the empirical research published between 2011 and 2017. Findings serve to demonstrate the diversity of this line of research, particularly in scale and scope of studies and in the approaches taken. By mapping the research using a systematic review methodology it is shown that there is a lack of qualitative data on how Twitter is used by learners and teachers in MOOCs. Moreover, a number of methodological gaps exist in published quantitative survey research at the interface between Twitter and MOOCs, including issues in the trustworthy reporting of results and full consideration of tweet and tweet meta-data collection. At the same time the paper highlights areas of methodological “best practice” in the research around these issues and in other important areas such as large-scale hashtag analyses of the use of Twitter in MOOCs. In reviewing the literature the findings aim to strengthen the methodological foundation of future work and help shape a stronger research agenda in this emerging area.

The extent of the research literature on Massive Open Online Courses (MOOCs) has mirrored the explosion in the number of such courses and learner enrolments. Systematic literature reviews that attempt to classify and synthesize this large new literature have been valuable to scholars in the field. For example, at the time of this writing, two of the early published reviews had accrued over 650 citations, according to Google Scholar (Liyanagunawardena et al. 2013; Veletsianos and Shepherdson 2016). The publication of major review articles on MOOCs continues to be an active line of research, as illustrated by recent studies providing a content analysis of trends and patterns in the literature (Bozkurt et al. 2017; Zawacki-Richter et al. 2018) and a systematic review of the research methods and topics investigated (Zhu et al. 2018).

The research landscape concerned with the application and implications of the microblogging platform Twitter for teaching and learning is similar. Here, systematic literature reviews have made a major contribution to scholarship, as evidenced by Gao et al. (2012) and Williams et al. (2013). However, no review on the intersection of these two research areas has been conducted to date based on the results of a systematic search of the major publication databases. The question arises as to what is the small talk about big courses and hence the current study was undertaken. The next section provides a brief overview of research literature that pertains to both Twitter and MOOCs in the context of learning and teaching to highlight the research gap(s) that this study seeks to address.

Microblogs, and Twitter in particular, promise that they can enable participatory, collaborative, and even playful forms of online (and mobile) learning (Dunlap and Lowenthal 2009). Given that use of the Twitter platform is free, and that most of its content is published openly online, it is not surprising that MOOC-using teachers have successfully adopted it as an alternative site of learning to augment their courses (Salmon et al. 2015; van Treeck and Ebner 2013). In some cases teachers have even taken to Twitter as the primary platform to engage learners and deliver the MOOC itself (Bozkurt et al. 2016). To-date studies have investigated a variety of topics, such as learner experience (Kop 2011), social capital acquisition Joksimović et al. (2015a, b) and Twitter’s mirror of the course unfolding inside the MOOC platform Joksimović et al. (2015a, b). It is noteworthy that the literature on the use of Twitter in the context of MOOCs is characterised by a range of research approaches but there is no systematic review of this to date.

Through the analysis of Twitter hashtags, researchers have conducted studies that extend beyond individual courses to explore large Twitter datasets comprising aggregates of many MOOC learners in multiple courses. For example, these studies have analysed MOOC learners according to the following foci: sentiment towards courses (Shen and Kuo 2015); temporality and learning Zhang et al. (2015); levels and types of discussion (Veletsianos 2017); and the relative influence of participants in conversations about MOOCs, including discussions on the phenomenon itself (Costello et al. 2017).

Given the scope and diversity of this research, a comprehensive and systematic review is required to build a deeper understanding of Twitter for MOOC learners and teachers and to help shape a research agenda in this emerging area. Therefore the current study explores this interface by systematically reviewing and synthesising the extant research literature that concerns both MOOCs and Twitter as an important research topic in its own right.

This study centres on the following broad research objective: To investigate the types of research design being conducted and published on the use of Twitter for teaching and learning in the contexts of MOOCs. To achieve this objective the study was framed around the following Research Questions (RQs):

RQ 1: How can the published literature be classified according to the methodological approaches followed by researchers?

RQ 2: What are the tools and methods researchers have reported using to collect and analyse twitter data?

RQ 3: What is the scope of the published research in terms of sample size, number of tweets and Twitter users, and study time frames?

RQ 4: What deeper theoretical assumptions define or underpin the adopted research methodologies?

RQ 5: To what extent have researchers studied Twitter in concert with one or more other social networks?

This study employed a systematic literature review methodology (Petticrew and Roberts 2009; Okoli 2015; Kitchenham 2004) using the most common and reputable online databases/indices i.e. Web of Knowledge, EBSCO, Google Scholar, Scopus and IEEE Explore. Systematic literature reviews are typically utilised to help categorise and better understand large bodies of information. They aid this understanding by mapping out major themes, areas of uncertainty and by identifying gaps in research that can indicate where further work is needed (Petticrew and Roberts 2009). A systematic literature review provides a clear protocol for a comprehensive search strategy, has explicit inclusion and exclusion criteria for studies, and a method of evaluating or synthesising findings. While there may be a risk of an overly narrow mechanistic approach if the analysis is not framed by an explicit theoretical lens, the major advantage and defining feature of the systematic literature review is that it clearly utilises a systematic and replicable methodology (Petticrew and Roberts 2009; Kitchenham 2004).

The inclusion criteria for this study comprised the rules the authors used to build the literature corpus: in other words, the first casting of the net. For inclusion, studies were required to meet five criteria: (1) examined the use of Twitter (or Sina Weibo) as a site for research on a MOOC or MOOCs, (2) empirical in their scope, (3) written in English, (4) published in peer reviewed journals or conference proceedings, and (5) published or available online between January 2011 and July 2017. For the basis of this study empirical research was defined according to a number of sub-criteria. Firstly, studies were required to have gathered primary data, i.e. conceptual, purely theoretical and opinion pieces were excluded. Data could be gathered from Twitter directly in the form of tweet content and/or tweet metadata, such as the network structures and characteristics of a Twitter community that discusses MOOCs, or indirectly, such as via interviews or surveys of perceptions of people who use Twitter for MOOC teaching, learning or research. The data collection instrument of the study needed to include a direct reference to Twitter or a tweet, i.e., a survey had to mention Twitter or tweets specifically. Thus, studies that asked participants about social media more generally were excluded.

Following Liyanagunawardena et al. (2013) and Veletsianos and Shepherdson (2016), we used the following search terms for Twitter, MOOCs and their academic synonyms (which also capture studies of Sina Weibo, the Chinese equivalent of Twitter):

(micro-blogging OR micro-blog OR microblogging OR Microblog OR twitter OR Tweet) AND (“Massive Open Online Course” OR “Massively Open Online Course” OR MOOC).

As indicated above the following databases/indices were searched to identify the published literature: Web of Knowledge, EBSCO, Google Scholar, Scopus and IEEE Explore. Of these sources, Google Scholar is known to index by far the largest amount of content, including non-peer reviewed work and other grey literature Haddaway et al. (2015). Therefore, following Gao et al. (2012), it was decided to search the full text of Google Scholar articles, sorting the search results by relevance and analysing the first two hundred results, at which point, we felt confident to have identified the most relevant results. The lead researcher conducted the search and the analysis of all the returned articles. There was of course considerable overlap between the databases. Most articles could be excluded from the study based on a reading of the abstract .e.g. whether it was an empirical study or just a thought piece. 41 articles need to be downloaded and read in full to determine if they met the criteria. This resulted in 34 eligible studies. We only include peer-reviewed studies which excludes “grey literature” such as dissertations, theses, working papers, unpublished papers blogs etc. Although the established practice for systematic literature reviews this is a prospective limitation especially if we consider that peer reviewed studies may be more likely to publish favourable outcomes.

Following the systematic search strategy outlined above and the application of the inclusion criteria, a dataset of 34 eligible articles was identified. Figure 1 below shows how many studies were published annually for each year for which full year data is available.

Fig. 1

Number of studies published for which data available for complete calendar year (n = 30)

The results of a detailed analysis of these articles according to our research questions are outlined in the next section.

At the top level we first classified the studies by dividing them into those that gathered perceptions data (n = 16) through survey i.e. self-reports via questionnaires or interviews and those that gathered primary Twitter data (n = 33). We further classified Twitter data studies as those that analysed tweets (n = 23) and those that analysed tweet metadata (n = 19) such as networks of Twitter users. Studies that analysed tweets included those in which a human evaluator qualitatively analysed tweets. The evaluator in such studies typically used a defined research instrument derived from a theory or other study, e.g., (Kop 2011) used social presence indicators. Our analysis showed that the evaluation was primarily based on content or socially orientated, e.g., whether a tweet refers to a course topic or an interaction with a learner/teacher. Other analyses of tweets (n = 22) used machine-learning approaches to analyse content. Machine learning approaches comprised those that detected themes via topic modelling and keyword mining Joksimović et al. (2015b) and/or sentiment towards particular topics i.e. sentiment analysis (Abeywardena 2014; Shen and Kuo 2015; Costello et al. 2016). Other machine-learning or computational approaches analysed the metadata of tweets, such as information on time periods of twitter activity, quantity of twitter activity and changes in twitter activity over time Zhang et al. (2015). A special category of tweet metadata was found to have analysed the connections between twitter users, such as their activity or whom they follow, known broadly as social network analysis (SNA). As per Table 1 below 12 studies conducted SNA.

It should be noted that researchers typically collected data from more than one data source e.g. all but one of the 16 studies on learner perception also combined this with analysis of tweet data in some way with a notable exception being the interesting study of Saadatmand and Kumpulainen (2014) which took a phenomenological approach purely based on participant testimony.

While a detailed enumeration of the studies comprising each category is provided in Table 1 above a typology of research approaches according to method is depicted in Fig. 2 below.

Fig. 2

A typology of research on Twitter and MOOCs

In terms of the predominant methods being employed only a minority of studies (n = 2) utilised specific instruments, such as pre-existing coding schemes for qualitative analyses of either tweets or learner responses to surveys or interviews (van Treeck and Ebner 2013; Veletsianos 2017). However, the majority of surveys simply asked questions about whether participants found Twitter easy to use or useful for learning without using an explicit theoretical reference point or an instrument derived from extant literature. These types of questions have some relation to constructs from the Technology Acceptance Model (TAM) literature; however, only one study specifically mentioned a model of this type (Koutropoulos et al. 2014). Lastly, one study (Salmon et al. 2015) stated that the researchers were not part of the MOOC teaching team, highlighting that few of the studies clarified this point. One study, Knox (2014), stated that the researcher was also a teacher. However, many studies implied that an overlap in the teaching and research teams, or at least their affiliations, existed.

How tweets were collected and analysed

RQ2 concerned the data collection methods employed in the research studies analysed and the prevalence of the various analysis methods employed.

A total of 16 studies reported using 11 different aggregators or methods for collecting tweets and tweet metadata, as listed in Table 2 below. Of these tools and methods, four were used in more than one study, namely, Crawler, NodexxL, Twinonmy and gRSShopper.

Table 2

Tweet Data and Metadata Collection Methods/Tools

Collection method/tool

Number of studies

gRSShopper

3

Crawler

2

NodexxL

2

Twinonomy

2

Twitter API

1

Crowdmap

1

Digital Methods Initiative Twitter Capture and Analysis Toolset

1

Search box on Twitter website

1

TagsExplorer

1

GNIP API

1

TwitterSTAT

1

As shown in Table 3 (below), 20 analytical and computational tools were reportedly employed for data storage, processing and analysis in studies that analysed tweets and/or tweet metadata. The tools mentioned in more than one study were R, Gephi, Excel, NVivo, NodexxL and Microsoft Translation API.

Scope of the studies

RQ3 examined the amount of primary data considered by researchers, the length of time the data pertained to and the numbers of tweets and twitter users. Seven of the studies either interviewed or surveyed the participants. The number of participants ranged from a minimum of 12 to a maximum of 3362, as outlined in Table 4 below.

A total of 13 studies reported the number of Twitter users that were included in the research where tweets or twitter metadata was analysed. The range in the size of the studies analysed was considerable, with the smallest study including 173 unique Twitter users and the largest including 278,685 users (see Table 5). The median value across the studies where this information was provided was 2431 users.

In terms of study timeframes some systematic literature reviews of Twitter in teaching and learning MOOC contexts have examined the length of intervention or, more generally, the timeframe in which data were collected (Gao et al. 2012;Williams et al. 2013). This timeframe may have been based on an individual MOOC or set according to a calendar year, for example. The shortest time frame studied was 2 weeks, while the maximum was that examined by Zhang et al. (2015), who collected 260 weeks’ worth of data. The median time frame was 10.5 weeks. The study time frames are outlined in Table 7 below.

In terms of the number of courses, the majority of the studies examined a single MOOC. Table 8 below, however, lists the studies that reported analysing data pertaining to more than one MOOC. The term “multiple” indicates that the researchers studied multiple MOOCs but did not specify an exact number (such as in studies of the #MOOC hashtag), which is somewhat unhelpful in judging the methodological trustworthiness of the research.

Theoretical assumptions

To answer RQ 4 an analysis was undertaken of the deeper theoretical assumptions or perspectives that researchers used within and across the 34 studies. The findings revealed a very diverse picture with little overall pattern. The theoretical assumptions or at times relatively light conceptual touchstones to indicate particular theoretical lens were not always explicit but when noted they included: Rhizomatic learning (Saadatmand and Kumpulainen 2014; Bell et al. 2016), Connectivism (Saadatmand and Kumpulainen 2014, Cruz-Benito et al. 2015), and social presence/Community of Inquiry (CoI) (Kop 2011; Enriquez-Gibson 2014a; Spilker et al. 2015; Bozkurt et al. 2016), with the latter cited most frequently (n = 5).

Social networks beyond twitter

Lastly, in terms of other social networks as per RQ5, 14 of the studies did not focus on Twitter or a MOOC in isolation but examined them in concert with one or more other social networks. Of the social networks other than Twitter encountered in this review, Facebook was the most prevalent, with Google+ also featuring prominently. Other social networks appeared less frequently, often with only one mention each. The social networks other than Twitter examined by the studies are shown in Fig. 3 below.

The amount of published research on the use of Twitter in MOOCs has increased, as shown in Fig. 1. This point thereby underscores the value of better understanding the scope and nature of this research body as a whole. As this is clearly a growing area of interest, this paper contributes to this branch of research by mapping out the current state of the field by collating, interpreting and summarizing the methods employed by extant published studies. Future researchers can use this mapping to situate their research interests or to identify gaps or under-researched areas. The following discussion section elaborates on some of these areas and generally reflects on the findings in relation to the research questions.

Firstly, the systematic literature reviews reveals a relative dearth of research utilising qualitative methods. For instance, only four of the 34 studies conducted interviews. In addition, the review highlights the breadth of this research area and the diversity of approaches taken–for better and worse. Our analysis of the theoretical underpinnings as per RQ4 suggests that researchers should be cognizant of existing theories and theoretical constructs, particularly when surveying MOOC Twitter learners where use of survey instruments derived from prior studies or established theory appear lacking. What this may highlight is that the field is in need of more theorizing to properly advance. It may be that the lure of this new, available and abundant site of research data has caused studies to be undertaken in a haste that did not allow for proper research design that would incorporate theoretical underpinnings. Our recommendation here is that authors pay heed to this in future and situate their work more explicitly with reference to relevant theory.

Secondly, in addressing RQ2, we analysed the data collection tools and methods employed in the research studies and the prevalence of the various analysis methods reported. As presented above many studies reported the tweet data and metadata collection methods they employed; however, four studies did not report the collection methods. Furthermore, many tweet collection methods employed screen-scraping or manual searching, which have methodolnogical implications for the reliability of the data, reproducibility of the studies and rights of the Twitter users (Driscoll and Walker 2014). Official Twitter APIs for example will remove deleted tweets from their datasets, respecting this right of users, or remove tweets from deceased people. Five of the studies included some critical analysis of tweet data collection, such as limitations of particular collection methods. One paper claimed to have a “complete corpus” of tweets (Bozkurt et al. 2016), while others contained discussions on how tweets can be harvested from Twitter and the relative limitations of such techniques, including the fact that only a sample can be retrieved (Koutropoulos et al. 2014; de Keijser and van der Vlist 2014). One study calculated that its tweet sample represented 80% of the underlying data (Veletsianos 2017). The key point is that a greater consideration and acknowledgement of the complex nature of tweet collection could be made in future studies in this area, and there is also scope for more studies using Twitter Streaming APIs and Big Data infrastructure.

On a related point Excel, Gephi, R, NVivo and NodexxL were the most commonly used analysis tools, as shown in Table 3, although a long list of other tools was employed suggesting a broad range of tools are being adopted. Most of these tools were used for SNA, but NVivo was also used for qualitative analysis. In the future the field would benefit from more explicit discussion around the advantages and disadvantages of particular analysis tools such as around their ease of use, sophistication and cost or availability. For example Gephi is a specialised tool for network analysis that can be used to readily create network statistics and visualisations. R by contrast arguably requires greater expertise to use but is a general purpose statistical platform which can conduct not only network analysis but multiple statistical and machine learning analyses and on potentially very large data sets via big data and cloud techniques. R and Gephi are open source software so free to download and use. The NodexL plugin is open source but the software it relies on, Microsoft Excel, is proprietary paid for software as is NVivo.

Thirdly, there also appears to be a gap in the research when we consider the scale of the survey research, as only one study reported a large (given the potential pool of participants) number of respondents, i.e., over three thousand respondents Alario-Hoyos et al. (2013). This issue is not helped, however, by the large proportion of studies (85%) that did not report explicit response rates, which suggests potential lack of rigor in at least the reporting of these studies. For instance, Bell et al. (2016) found greater levels of discussion of a MOOC on Facebook than on Twitter.

Another related point is that the size of scope of the studies varied widely. Indeed, an alternative mapping of this literature might examine study size. Nine of the studies analysed over 10,000 tweets. Manual qualitative evaluation at this scale becomes difficult in a practical sense for all but relatively small samples (Veletsianos 2017) i.e. it would be prohibitive for a researcher to manually read and classify thousands of tweets. However, the relationship between data scale and practical study methods is not symmetrical, i.e., machine-learning techniques can be used in small-scale studies; thus, we employed the study mapping approach based on method type. Another small but interesting subcategory of the research studies consists of those that considered multiple MOOCs. A total of 15 studies were conducted on more than one MOOC, but several analyses were performed on over 100 MOOCs (Shen and Kuo 2015; Tu 2014; Zhang et al., 2015; Kravvaris et al. 2016; Veletsianos 2017; Costello et al. 2016; Costello et al. 2017). The variance of the dataset sizes in these studies has implications for the comparability of findings. Future studies may need a stronger justification for the use of particular analytic approaches taken.

Fourthly, the study timeframes over which research was conducted generally mapped to course lengths. However, considering a window that stretches beyond the course could be valuable for researchers and course designers, such as the approach used by Bozkurt et al. (2016), which showed Twitter activity up to 3 weeks after the course. The findings on study length also mapped the trend in shorter MOOC durations as course lengths have been shown to inversely correlate with completion rates (McIntyre 2016). The work of Zhang et al. (2015) is notable for its examination of a large body of MOOC learners (and other stakeholders) with regard to the temporal dimension of their Twitter activity related to MOOCs. They noted, for instance, peaks of activity during particular times of the year and week. This is an underexplored aspect of research and highlights the significant potential for studies of large datasets of the Twitter activity related to multiple MOOCs. Indeed, a few studies belonged to a special category that analysed the hashtag #MOOC itself (Abeywardena 2014; Shen and Kuo 2015; Zhang et al. 2015; Costello et al. 2016; Costello et al. 2017). This hashtag can be used to create large datasets that contain not only learners and teachers but also researchers, platform providers and other MOOC stakeholders.

Fifthly we found that Facebook was the second most common social network researched after Twitter. It should be born in mind however, some studies were excluded from our analysis at the dataset creation stage because they did not employ separate questions for Facebook and Twitter use even though it could be argued that they are quite different media. Hence we recommend that researchers are clear in the framing of their questions and reporting of their results where possible and appropriately disaggregate data derived from different social networks.

Finally, some further interesting possible analyses were beyond the scope of our original research questions. For instance we did not examine the issue of ethics, data protection and ethical approval. This could be usefully examined in a future study. Another interesting analysis that was beyond the scope of the current study would be to examine in detail the specific research questions that the studies used and how these related to the topics under analysis. For future researchers, we also recommend that they try to include as much details of their approach, their data and analyses as possible to help facilitate reviews of this research. Moreover in the five points above we include examples where we believe best practice has been shown.

The media hype surrounding MOOCs may have somewhat abated but interest from learners continues to grow. While the digital footprints left by these learners and their teachers is usually analyzed within the big course learning environment of MOOCs the small talk of learners spreads outwards via Twitter in a myriad of ways. As demonstrated in this systematic literature review the variance in the scope and scale of studies exploring the interface between Twitter and MOOCs suggest that future researchers will do well to carefully justify their approaches and consider some of using qualitative methods to analyse appropriate research problems. Finally, by critically synthesising and developing a typology of the literature in this area, this study has hopefully provided an agenda for this growing area of research and contributed signposts and jumping-off points for future work.

Acknowledgements

The authors would also like to thank Beijing Normal University Big Data Centre for Technology-mediated Education and the National Institute for Digital Learning and in Dublin City University. Ms. Virginia Thomas is acknowledged for her role in preparation of the manuscript.

Funding

This work was supported by the Chinese National Education Science Foundation [Project No. CCA120110].

Availability of data and materials

The data that support the findings of this study comprise academic papers are available from the respective publishers. We include full references to all of the papers in this article text.

Declarations

The study involved no human participants. All information, comprising the articles and papers analyzed, has not been used for purposes other than originally intended i.e. to contribute to scholarly research. There is no conflict of interest. The authors have included some of their own work in the analysis according to the criteria outlined in the article.

Authors’ contributions

The first author conducted the primary analysis which was validated by the second and third authors. All authors contributed to the literate review and read and approved the final manuscript.

Competing interests

The study involved no human participants. All information, comprising the articles and papers analyzed, has not been used for purposes other than originally intended i.e. to contribute to scholarly research. There is no conflict of interest. The authors have included some of their own work in the analysis according to the criteria outlined in the article.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Bozkurt, A., Akgun-Ozbek, E., & Zawacki-Richter, O. (2017). Trends and patterns in Massive Open Online Courses: Review and content analysis of research on MOOCs (2008–2015). International Review of Research in Open and Distributed Learning, 18(5), 118–147.View ArticleGoogle Scholar

Zawacki-Richter, O., Bozkurt, A., Alturki, U., & Aldraiweesh, A. (2018). What research says about MOOCs – An explorative content analysis. International Review of Research in Open and Distributed Learning, 19(1), 242–259.View ArticleGoogle Scholar

Zhang, J., Perris, K., Zheng, Q., & Chen, L. (2015). Public response to “the MOOC movement” in china: Examining the time series of microblogging. The International Review of Research in Open and Distributed Learning, 16 (5), 144–159.Google Scholar