Abstract

Objectives

To investigate study quality and reported prevalence among the emergent area of problematic internet use (PIU) research conducted in populations of US adolescents and college students.

Data sources

We searched PubMed, PsychInfo and Web of Knowledge from inception to July 2010.

Study selection

Using a keyword search, we evaluated English-language PIU studies with populations of US adolescents and college students.

Main outcome measures

Using a quality review tool based on the STROBE statement, two reviewers independently extracted data items including study setting, subject population, instrument used and reported prevalence.

Results

Search results yielded 658 manuscripts. We identified 18 research studies that met inclusion criteria. Quality assessment of studies ranged between 14 and 29 total points out of a possible 42 points, the average score was 23 (SD 5.1). Among these 18 studies, 8 reported prevalence estimates of US college student PIU, prevalence rates ranged from 0 to 26.3%. An additional 10 studies did not report prevalence.

Conclusion

The evaluation of PIU remains incomplete and is hampered by methodological inconsistencies. The wide range of conceptual approaches may have impacted the reported prevalence rates. Despite the newness of this area of study, most studies in our review were published over 3 years ago. Opportunities exist to pursue future studies adhering to recognized quality guidelines, as well as applying consistency in theoretical approach and validated instruments.

INTRODUCTION

Internet use is nearly ubiquitous among adolescents and young adults; current US data suggests that 93% of adolescents and adults between the ages of 12 and 29 years go online.1 Given these high rates of internet use, internet addiction, often described as “problematic internet use which is uncontrollable and damaging,” is a growing concern.2, 3 Several studies in the US and abroad, and numerous anecdotal media reports, suggest possible links between overuse of the internet by adolescents and young adults and negative health consequences such as depression, ADHD, excessive daytime sleepiness, problematic alcohol use, or injury.4–8 Internet addiction has also been associated with negative academic consequences such as missed classes, lower grades and even academic dismissal.9–11 Currently, internet addiction is proposed as a disorder in need of further study for the appendix of the Diagnostic and Statistical Manual V (DSM-V).12

Efforts towards developing diagnostic criteria for internet addiction or problematic internet use (PIU) began in the 1990s. Two initial approaches to PIU were based upon existing DSM-IV disorders: substance abuse/dependency and pathologic gambling.13, 14 This early work was accompanied by the introduction of three conceptual approaches. First, PIU was more broadly described as a general behavioral addiction. 15, 16 Second, a cognitive-behavioral model of PIU drew attention to the impact of an individual’s thoughts on their development of problematic behaviors, and separated PIU into “generalized” PIU, or multidimensional overuse of the internet, and “specific” PIU.17 Specific PIU was defined as dependence on a specific function of the internet. Third, a model proposed that PIU should be more widely classified as a impulse control disorder with criteria defined as: a) maladaptive preoccupation with internet use characterized by either irresistible use, or use that is excessive and longer than planned; b) clinically significant distress or impairment; and, c) an absence of other, explaining, Axis I disorders.18 These differences in the conceptual approach towards PIU have influenced the various instruments that have been developed to evaluate PIU.

At present, there are at least 13 instruments designed to measure PIU. Several were adapted from the DSM-IV substance abuse and dependency criteria, such as the Internet Addiction Disorder Diagnostic Criteria19 and the Internet-Related Addictive Behavior Inventory.20 Others are based on the DSM-IV criteria for pathological gambling, including the Young Diagnostic Questionnaire 14 and Young Internet Addiction Test (IAT)21 (the latter being an expansion of the former), the Chen Internet Addiction Scale,22 and the Problematic Internet Usage Questionnaire.23 Other instruments are based on the PIU behavioral addiction model, such as the Compulsive Internet Use Scale24 or the Griffith Addiction Components Criteria.25 Additional instruments are based on the Davis cognitive-behavioral model of PIU, including the Online Cognition Scale (OCS)26 and the Generalized Problematic Internet Use Scale (GPIUS).27

Given the high rates of internet use among adolescents and young adults globally, it may not be surprising that research on PIU in this population has received intense international attention. Prevalence estimates of PIU vary widely. In studies focused on adolescents, European prevalence estimates are reported as between 1–9%,28–32 Middle Eastern prevalence estimates are between 1–12% 33–35 and Asian prevalence estimates are reported between 2–18%. 36–43 Similarly, the prevalence for international college students has been reported as between 6–35%.44–47 It is unclear whether the wide range of prevalence estimates reported is related to cultural differences between regions or countries, or due to different approaches in the definition and assessment of PIU.

Despite the timeliness and importance of this topic, to our knowledge, a systematic review of the existing literature on PIU among US adolescent and college students examining both study quality and reported prevalence is lacking. As research findings often lead to diagnostic criteria and clinical practice, the quality of such studies is of the utmost importance. Our goals are to examine: 1) the quality of studies in this area, and 2) the prevalence rate for problematic internet use among US adolescents and college students. By conducting this systematic review we provide an understanding of the current approaches to PIU and a framework upon which future research endeavors can be built.

METHODS

Search strategy

In consultation with a health sciences librarian, a systematic review was performed of three major databases incorporating medical and social science literature. PubMed, PsychInfo and Web of Knowledge were searched from inception to July 2010. As no Medical Subject Headings (MeSH) terms were found to fit our topic of interest, we identified key word search terms starting with the terms “internet addiction” and “problematic internet use” and building additional terms by identifying keywords associated with those searches or within articles found in those searches. A final list of search terms included the following keywords or keyword combinations: internet addiction, compulsive internet use, problematic internet use, pathological internet use, internet dependence, and excessive internet use. To identify additional articles that addressed problematic internet use we searched the bibliographies of included studies.

Study selection

Given the current consideration of internet addiction for inclusion in the DSM V, we chose to focus our review on studies which investigated internet use as a source of addiction or dependency. We did not investigate related concerns, such as inappropriate use of the internet for sharing sexual explicit material or cyberbullying. Thus, we included English-language studies that (1) involved a US population, (2) focused on adolescents or college student participants, and (3) assessed internet addiction symptoms empirically through the use of a scale or set criteria. We excluded non-US papers, studies that focused on adults, studies that did not assess PIU specifically, non-empirical work such as case studies or commentaries and unpublished literature. Searches were initially screened for inclusion using titles of articles and abstracts when available, when inclusion criteria were not clear from the title and abstract the full text was evaluated. Full text of articles that met inclusion criteria were retrieved and systematically assessed by two investigators.

Quality Review Tool

The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement delineates essential items to be reported in observational research studies.48 At present, a specific tool for assessing the quality of problematic internet use studies is lacking. In order to assess the quality of PIU studies reporting prevalence data, we developed a Quality Review Tool (QRT), deriving our items from the STROBE statement.48 (Table 1) The QRT developed for this review consists of 21 items that assess the quality of study design, data collection and analysis on the basis of reported information. Each item was scored a maximum of 2 points if full reporting criteria were met, 1 point if partial criteria were met, and 0 points if no reporting on was present, for a total possible score of 42 points. Two investigators (MM and LJ) scored all articles. Score discrepancies were rare (QRT total scores identical >85% of the time), any discrepancies were resolved by consensus.

RESULTS

Our electronic search yielded 658 total references, 396 of which were initially eligible based on their publication in English in a peer-reviewed journal. (Figure 1) Of excluded studies, 137 were not conducted in the US, 42 were not focused on adolescents or college age populations, 65 did not focus on problematic internet use (i.e. focused on instant messaging addiction, pornography addiction, computer gaming addiction) and 134 were not empirical studies. Among the remaining studies, 8 were determined to have used a problematic internet use/internet addiction screening instrument and reported PIU prevalence estimates,23, 49–55 and 10 used an instrument but did not report prevalence (Table 2).27, 56–64Table 2 presents data from each study included in the systematic review; studies are organized based on the conceptual approach of the PIU assessment utilized. All studies focused on college student populations, we found no studies specifically targeting adolescent populations.

Quality

A total of 8 studies that provided descriptive data and reported prevalence were assessed using the QRT. Quality assessment of studies ranged between 14 and 29 total points out of a possible 42 points, the average score was 23 (SD 5.1). The majority of these studies received less than two-thirds of the available 42 total quality points (Table 3). Individual QRT categories that occurred least frequently across all studies included: explanations for the selected sample size (0 out of possible 16 total points), response rate reporting (2 out of 16 total points); study timing reporting (3 out of 16 total points), rates of missing data (3 out of 16 total points). The item that measured use of a piloted or validated instrument scored only 5 out of a possible 16 total points. Only 3 studies reported ethnicity (5 points out of possible 16 total points). Only 1 study documented rates of missing data (2 out of possible 16 total points).

Individual QRT categories that occurred most frequently across all studies included describing the recruitment strategy (16 out of 16 points) and describing statistical methods used (16 out of 16 points).

Prevalence of PIU

Overall, the range of prevalence of PIU in examined studies was between 0% and 26.3%. The reported prevalence of PIU must be considered in the context of the conceptual approach identified in that study (i.e. substance use, pathological gambling).

Four studies evaluated PIU based on DSM IV criteria for substance use. Three of these studies defined “internet dependency” as a participant answering affirmatively to between 3 and 4 items out of 7 to 10 total items; these studies found that prevalence ranged from 9.8 to 15.2%.49, 52, 54 The fourth study used both a “liberal” and “conservative” set of criteria to determine criteria for both internet abuse and dependency. This study found a range of 1.2 to 26.3% prevalence for dependency within a single sample.50 A single study used the IAT, based on DSM IV criteria for pathologic gambling.23 This study defined internet addiction as scoring over 40 total points and found a prevalence of 25%.51

Three studies used independently generated instruments without a specifically described conceptual model and found prevalence to between no participants meeting criteria and 12.6%.53, 55, 62 Among these, one study conducted assessments in two populations. No estimate was given for overall prevalence for the first sample, although reference was made to participants meeting criteria, while no participants met the criteria for PIU in the second sample.55

Studies that did not report PIU prevalence rates

Among the 10 studies that did not report prevalence estimates, the majority were focused on developing a conceptual model of PIU or validating an instrument scale. These studies used a range of instruments, some of which were independently developed, as well as the IAT, the OCS, and the GPIUS. Of these ten studies, three introduced and validated new instruments,27, 63, 64 two adapted previously validated instruments,23, 60 and five modified previously validated instruments, which included the use of additional items.56–58, 60, 61

CONCLUSION

Overall our findings suggest a paucity of empirical studies addressing PIU among populations of US adolescent and college student populations. Despite initially finding over 600 search hits on the topic of PIU, only 18 articles were identified that met inclusion criteria, less than half of these reported a prevalence estimate. We found no studies specifically targeting adolescent populations.

Among these studies, the overall quality scores were very low. Many of the QRT items that received particularly low scores, such as using a validated instrument and reporting missing data, have significant impact on the internal validity of the findings. Further, other areas that received low scores, such as reporting response rates and participant characteristics, critically impact the external validity of these studies. Future studies of PIU could consider using the STROBE criteria or our QRT to enhance the quality of the study and thus the validity of the findings.

The studies examined in this review reported prevalence rates ranging from no participants meeting criteria to up to a quarter of participants meeting criteria for PIU. There are several possible reasons that this range of reported prevalence rates is so wide. First, many of these instruments applied vastly different conceptual approaches based on addictions such as substance use or gambling, or other cognitive, behavioral or impulse-control models. The lack of consensus in conceptual approach to PIU may be a key reason for the variability amongst these studies approach and findings. Second, perhaps related to the lack of consensus on the appropriate conceptual approach to PIU, the majority of studies in this review used independently created instruments whose conceptual framework is incompletely evaluated. This then leads to additional challenges because the psychometric properties of these new instruments are often incompletely evaluated. Third, instruments used to evaluate PIU applied varying response mechanisms: some used Likert scales which allow for reporting the degree and severity of symptoms or consequences, and others used binary “yes/no” responses which may not fully capture the frequency or severity of a problematic behavior. Fourth, the cut-offs for criteria defining the when a participant met criteria for PIU varied among the instruments used to assess PIU. As studies did not correlate their cut-points to actual negative consequences such as behavioral or achievement problems, it is difficult to know whether participants who were labeled as having PIU were actually experiencing any offline consequences.

Last, over half of the studies reporting prevalence estimates were conducted over five years ago during a time where wide-scale internet use was still varied and growing. Immense changes in both internet access and use have occurred over the last decade.1 Thus, it is reasonable to assume that not only the extent of, but also the populations most at risk for, internet addiction may have changed from what was evident in the past. More recent work is required to determine not only a current estimate of prevalence based on a standardized approach, but also what characteristics may put an individual at increased risk in our current technology-saturated culture. Findings which are informed by current internet use standards and trends may also help to shape the development and definition of a diagnosis for a clinical disorder.

The findings in this review may be limited as we did not search the gray literature (evaluation of theses, dissertations or unpublished work). However, many of the studies examined in our review had methodological flaws limiting external validity, such as failure to report response rates, thus the gap between unpublished and published literature may be small. Further, given the newness of this field and the wide range of prevalence rates reported in studies, including studies that reported a prevalence rate of 0%, it is likely that publication bias may also be small. Our goal in this study was to evaluate US studies, thus, generalization beyond the US is not warranted.

Despite these limitations, our study findings illustrate the critical need for additional rigorous study of PIU. However, in order to fully understand and estimate the impact of this new disorder, we must first have consistency and consensus in the approach to its assessment. Among the instruments identified in this study, the IAT was the only validated instrument used in a study that reported prevalence rates. Another validated and frequently used instrument was the OCS, although this scale was not used in studies reporting prevalence data. Thus, these instruments may be a useful starting point for future study. As both of these measures were initially developed over 8 years ago, re-evaluating their construct structure and establishing face validity in the context of today’s internet-rich environment and within this target population will be an important initial step. Administering multiple instruments in the context of a single study to determine overlap and concurrent validity may be useful in the pursuit of developing a comprehensive instrument to assess PIU. Following this, further rigorous studies using a validated instrument and incorporating recognized quality criteria may be conducted to confirm prevalence data. Finally, among studies that reported time spent on the internet, all relied upon participant self-report for cumulative internet use. Future studies that provide more accurate means of measuring internet use are needed.

Further, of note, no US studies identified in this review included samples focused on the adolescent population, and studies of college students were generally limited to a single university and modest sample sizes. Future large-scale studies within these at-risk populations are urgently needed to confirm and enhance generalizability. Several European and Asian countries have included assessments of internet addiction within national assessments of adolescent and college student health.10, 28, 65, 66 Adopting similar methods within the US may allow for accurate identification and estimated scope of this problem on a national level.

If internet use has potential to lead to addiction, this means that up to 93% of US adolescents and young adults are exposed to this risk, dwarfing exposure rates for any other behavioral or substance-based addiction.1 Before we can fully understand this important phenomenon, we must first have consistency and consensus in the approach to its assessment. Only after these studies have firmly established current prevalence and considered risk factors, can we make informed considerations on what diagnostic criteria should be recommended for inclusion within the DSM or how to evaluate the successes of any proposed treatment programs.

Acknowledgments

Support for this project was provided by award K12HD055894 from NICHD. The authors would like to thank Heidi Marleau for her assistance with this project.

41. Wu J, Lin G, Lin L. Analysis of the situation of internet use and the related health-risky behaviors among the youngsters in Guangzhou City. Journal of Tropical Medicine (Guangzhou) 2007;7(8):816–818.

45. Niemz K, Griffiths M, Banyard P. Prevalence of pathological Internet use among university students and correlations with self-esteem, the General Health Questionnaire (GHQ), and disinhibition. Cyberpsychol Behav. 2005 Dec;8(6):562–570.[PubMed]