Teaching basic lab skillsfor research computing

Our IUSE Proposal Was Rejected

We got word a few days ago that our proposal to the
NSF's Improving
Undergraduate STEM Education program had been rejected.
The panel summary agreed that software training is a good idea,
but were not convinced about our plans to shift from training grad students to undergrads.
In particular,
they were not convinced that Unix-based workshops would be best for undergrads,
and felt that not being embedded in the regular curriculum was a weakness.
These are fair criticisms:

Most undergraduates only use GUIs and cannot navigate the terminal to save their lives.
The perception that Software Carpentry is Unix-specific therefore hurt the proposal,
and showed that we didn't clearly explain our focus on underlying concepts.

The fact that we wouldn't be part of the regular curriculum is more difficult to address.
The fact is,
it really is hard to get undergrads to do things that interfere with earning grades,
just as it's often hard to get grad students to do anything that doesn't immediately lead to a publication.
Some of their concern about impact also seemed to be due to us concentrating on REU students,
a self-selected bunch that already are motivated.

At the same time,
though,
the panel did not connect the results from years of Software Carpentry workshops
and the expected impact of this effort.
This indicates that the proposal did not effectively communicate how well our experience to date
has laid the groundwork for efforts like this.

We're obviously disappointed by this rejection,
but we've some useful lessons,
and we hope that they will inspire others to put forward proposals of their own.

The Problem: Computational Competence in Science

Scientists and engineers invented electronic computers to accelerate
their work, but two generations later, many researchers in science,
technology, engineering, and mathematics (STEM) are still
not computationally competent: they do repetitive tasks
manually instead of automating them, develop software using a
methodology best summarized as "copy, paste, tweak, and pray" and
fail to track their work in any systematic, reproducible way.

While the World-Wide Web was created by a scientist to help his
peers share information, many still use it primarily as a way to
find and download PDFs. Researchers may understand that open data
can fuel new insights but often lack the skills needed to create and
provide a reusable data set. Equally, any discussion of changing
scientific publishing, making research reproducible, or using the
web to support "science as a service" must eventually address the
lack of pre-requisite skills in the general STEM research community.

Studies have repeatedly shown that most researchers learn what they
know about computing by word of mouth, but this approach is failing
to meet present-day needs: most faculty would agree that today's
graduates are no more able to use computing and the web in their
research than they were a generation ago. Attempts to integrate
more training in basic computing skills into undergraduate education
have largely failed to take root for several reasons:

The curriculum is full. Undergraduate STEM programs
already struggle to cover material regarded as core to their
field. While many scientists would agree that more material on
programming, reproducible research, or web-enabled science would
be useful, there is no consensus on what to take out to make room.

The blind leading the blind. Many faculty lack
computational skills themselves and hence are unable to pass them
on.

Cultural difference. Scientists and software developers
have different priorities and different approaches to problem
solving, which often impedes collaboration and knowledge transfer.

One final issue is that the rewards are unknown. Open,
web-based science is still in its infancy, so there is no general
understanding of what people might need to know in order to
incorporate it into their research careers. Since it is hard to
measure something if you don't know what to look for, or if it is so
young that there hasn't actually been long-term impact,
little systematic study has been done to date of whether early
training in the skills needed for this new kind of science actually
has an impact, and if so, how and how much. Without such feedback,
there is no systematic way to improve the training programs that
currently exist.

This proposal builds on the success to date of the Software
Carpentry workshops, a proven curriculum of essential software
skills that enhance the productivity of graduate students,
post-docs, and faculty. We propose to:

conduct formative evaluation of the impact of software skills
training for undergraduates likely to continue in research careers
as they progress through the early stages of those careers;

conduct summative evaluation of the training's overall impact on a
multi-year timescale in order to improve the content and
presentation of the training; and

disseminate the resulting curriculum widely.

We will run software-skills workshops for undergraduate students
taking part each year in summer research opportunities such as the
NSF's Research Experience for Undergraduates (REU) program, at or
near the start of those students' time in the lab. Based on data
already collected from Software Carpentry workshops, we expect this
training will help them be more productive during their research
(graduate-level participants in our existing workshops typically
report that what we teach saves them a day per week) and will
prepare them to work in a world where all aspects of science are
increasingly dependent on computing.

These undergraduates will serve as the treatment population for a
five-year study of the impact of this training on their careers in
general, and their involvement with open and web-enabled science in
particular. In order to conduct this study, we will hire an expert
in educational assessment, whose full-time work for the duration of
the project will be to explore the effects of the training on
workshop participants.

Workshops: A Distributed Model for National Impact

We will run two-day workshops at a steadily increasing number of
sites each year for five years, timed to coincide with the start of
the summer influx of undergraduate research students. Each workshop
will be offered to a minimum of 40 learners per site, giving us a
target study population of at least 2200 students by year 5. The
content will be tailored to meet local needs but will be based on
what is being used at that time by Software Carpentry and affiliated
educational efforts. By design, it is straightforward to adapt
workshop materials and contribute changes to the Software Carpentry
course material. These features enhance the portability and
flexibility of the workshops and increase the likelihood of wide
dissemination beyond this project.

The home sites for investigators named in this proposal (George
Washington University, Michigan State University, University of
California, Berkeley, University of Wisconsin – Madison, and
Utah State University) will run workshops in each of those
years. Other sites will be added each year, expanding the total to
15 in year 5 and accordingly increasing the size of our study
population. We will focus expansion on NSF REU sites but, as
detailed below, we will also offer some workshops to other
communities.

One set of possible sites for expansion are those campuses included
in the "Condo of Condos" consortium, recently recommended for
funding by the National Science Foundation. The Software Carpentry
workshops proposed here will be valuable to that consortium in
meeting its goals of increasing the number and diversity of
researchers using advanced cyberinfrastructure and of developing
data science practitioners.

Beyond this consortium, we will recruit sites for hosting workshops
by identifying locations at which we could have the largest impact
and/or that contribute most to our goal of increasing diversity. If
we find there are more sites interested in hosting workshops than we
are able to support in a given year, we will select the subset of
sites that best meet our goals.

Curriculum: From Tools to Techniques to Concepts

While there is considerable scope for customizing workshops to
accommodate learners' prior experience and discipline-specific
needs, what these workshops seek to convey is the best practices a
researcher needs to be computationally competent:

how to create, use, and share structured data

how to automate repetitive tasks;

how to track and share work over the web; and

how to grow a program in a modular, testable, reusable way.

With these objectives, the base workshop format will be divided in
to four modules. All workshops will be hands-on so that learners can
"learn by doing" and have experience with concrete examples. A
typical workshop will devote roughly half of a day to each of the
following:

Working with Data: this module will introduce students to
efficient data manipulation. Learners will work with data
representative of their field of study. By the end of this module,
learners will have the basic skills for parsing data, using and
structuring databases, and conducting more sophisticated
statistical analyses. We will point out why spreadsheets are
insufficient for these types of tasks.

Automation with the Unix Shell: this module will introduce
learners to the shell; teach how to view, search, and manipulate
text files at the command line; and introduce basic automation at
the command line.

Structured Programming: this module teaches learners introductory
Python or R. Learners will be able to write short scripts and work
in IPython Notebook, RStudio, or similar environments. This module
will also teach key ideas—iteration, conditional statements,
and modularity—that are essential for computational
competence.

Version Control and Data Sharing: this module teaches learners how
to keep track of their code, data, and analyses in an open and
reproducible way. The lessons will focus on data management
strategies as well as provide an introduction to GitHub for
version control of scripts, programs, analyses, etc. The real
lessons here are about conducting open, reproducible research.

As the module descriptions suggest, our real aim isn't to teach
Python, Git, or any other specific tool: it's to
teach computational competence. We can't do this in the
abstract: people won't show up for a hand-waving talk, and even if
they do, they won't understand. If we show them how to solve a
specific problem with a specific tool, though, we can then lead into
a larger discussion of how scientists ought to develop, use, and
curate software.

These workshops strive to show people how the pieces fit together:
how to write a Python script that fits into a Unix pipeline, how to
automate unit tests, etc. Doing this gives us a chance to reinforce
ideas and also increases the odds of participants being able to
apply what they've learned once the workshop is over.

Execution: Quality Instruction

We will aim for no more than 40 people per room at a workshop, so
that every learner can receive personal attention when needed. Where
possible, we will run two or more rooms side by side and use a
pre-assessment questionnaire as a sorting hat to group learners by
prior experience, which simplifies teaching and improves their
experience.

All of the workshop instructors will have been trained and certified
by Software Carpentry and will have had experience teaching this
material prior to engaging in these particular workshops. Just as
importantly, instructors will themselves be working scientists. By
virtue of using these skills and concepts daily in their own
research they are better able both to serve as role models and to
deal with unanticipated questions or challenges based on personal
experience.

Software Carpentry has a rich network of trained instructors, often
recruited from past workshops. These instructors are volunteers who
participate for a variety of reasons including sharpening their own
teaching and computing skills, increasing diversity in the pipeline,
and because it's fun.

As well as instructors, we will rely on local helpers to wander the
room and answer questions during practicals. These helpers may be
participants in previous workshops who are interested in becoming
instructors, graduate students who've picked up some or all of this
on their own, or members of the local open source community; where
possible, we will aim to have at least one helper for every eight
learners.

In order to increase the diversity of the study population, at least
one workshop in each year will be aimed specifically at female
students. Software Carpentry's first such workshop, held in Boston
in June 2013, attracted 120 participants; its second is scheduled
for Lawrence Berkeley National Laboratory in April 2014, and at
least two more will be held by the time work on this project
commences (one in the United States and one in Europe). This work
will build on that experience and draw on the pool of instructors
who have gained mentoring experience through those specific
workshops.

Finally, we will organize workshops in years 2 through 5
specifically aimed at students from minority serving
institutions. We are already in contact with the Computing Alliance
for Hispanic-Serving Institutions (CAHSI) and with the Association
of Public and Land-grant Universities' program for historically
black colleges and universities (HBCUs). Software Carpentry is
running its first workshop at an HBCU (Spelman College) in early
2014, and we expect to have significantly expanded these efforts by
year two of this project.

Formative and Summative Assessment: Maximizing Learning and Impact

We will employ an expert in educational assessment full-time for
five years to monitor and compare undergraduate participants in
these software carpentry skills building workshops, participants in
a subset of our regular (graduate-level) workshops, and
non-participants (as a control population). As part of their work,
this person will be responsible not only for collecting and
analyzing data but also for refining and extending the methods and
measures used to gauge impact. D-Lab will assist in locating and
supporting this expert.

Assessment will build on previous work, focusing particularly, but
not exclusively, on the following questions:

Are students who receive this training more likely than their
peers to develop new tools and practices and/or become involved in
outreach and education activities (i.e., are they more likely to
become creators and leaders)?

Are students who receive this training more likely than their
peers to incorporate open science and/or web-enabled science tools
and practices into their work?

Do outcomes differ between women and underrepresented minorities
on one hand and non-underrepresented minorities and men on the
other? If so, in what ways, and what steps are effective in
correcting for these differences?

In what ways does this training change students' outlook on the
practice of science itself?

Are students who receive this training more likely than their
peers to choose computationally oriented research topics and/or
careers? Are those who do not choose computationally oriented
paths nevertheless more likely to incorporate the tools and
practices mentioned above into their work?

Are students who receive this training more likely to continue to
graduate school than their peers?

This expert in educational assessment will explore ways in which our
engagement with students changes the outlook and work practices of
their peers and faculty supervisors (i.e., whether there is
knowledge transfer sideways and upward) and the effectiveness of the
community building and dissemination activities detailed in the next
sections.

As with Software Carpentry's work to date, assessment will use both
qualitative and quantitative techniques. On the qualitative side, we
will conduct a series of interviews over the five-year period of the
study to see how attitudes, aspirations, and activities
change. Quantitatively, we will measure uptake of key tools such as
version control as a proxy for adoption of related practices, as
well as exploring more traditional measures of research success,
such as progression to graduate school and publication/citation
rates.

To track impact over time, we will conduct pre-workshop surveys and
interviews roughly in the week before the workshops and the series
of post-workshop surveys and interviews beginning approximately one
month following the workshop. Follow on surveys and interviews will
be conducted and the end of the students' lab appointment and
annually thereafter (as applicable based on year of participation in
the project).

Our findings, and any new methods or measures developed, will be
shared with other researchers through publication in peer-reviewed
journals and high-profile conferences.

Community Building: Supporting Computational Competence

We will employ one graduate student part-time at each site named in
this proposal each year to provide technical support to workshop
participants, and to act as an anchor for a Hacker Within-style
grassroots group at that site. These community liaisons will not be
study subjects but will help us stay in touch with students who are
(a key requirement for any longitudinal study).

Separately, the Mozilla Science Lab will focus part of its ongoing
community engagement efforts on the students who have taken part in
our workshops during both the remainder of their undergraduate
careers and afterward to help them become part of the broader open
science community. This may include helping the students organize
and run workshops of their own in subsequent years, connecting them
with other open science projects, introducing them to potential
graduate supervisors who understand and value their new skills and
outlook, etc.

As a subordinate part of their work, the assessment researcher
employed by this project will assess the effectiveness of the local
graduate student organizers. In particular, they will explore
whether seeding activity in this way leads to the formation of
self-sustaining grassroots groups, and if so, what activities those
groups develop on their own, how (and how effectively) they share
discoveries with each other, the extent to which alumni of this
program stay engaged with these groups, and whether the presence of
these groups has a demonstrable impact on students' career paths in
general and/or on their engagement with open and web-enabled science
in particular.

While it will not be feasible to bring all of the students
participating in a given year's workshops together physically, we
will organize and run virtual conferences toward the end of their
research term to give them an opportunity to present their work to
one another, discuss what they have learned and build peer-to-peer
connections. These conferences will also provide an opportunity to
introduce participants to new forms of scientific "publishing",
including blogging, the creation of screencasts and demonstration
videos, and other methods that may not yet exist.

Curriculum Development and Dissemination: Expanding the Impact

We will employ one instructional designer part-time throughout the
project to create new material and to improve existing material
based on feedback from workshop participants and the assessment
program. Here, "creating material" may include both designing and
implementing new domain-specific learning modules and translating
existing materials into new forms, such as video recordings of
lectures or auto-graded quizzes for self-paced instruction. This
work will be done in consultation with educators at participating
institutions in order to encourage incorporation of those materials
into existing curricula.

All of the materials produced by and for this project will be made
freely available under the Creative Commons – Attribution
(CC-BY) license. The instructional designer will work with the
Mozilla Science Lab and affiliated groups to share these materials
and the results of our studies of the program's impact, through
science education journals, conferences, and other channels.

The dissemination of this project's curriculum has strong potential
to be high. Current Software Carpentry materials are available as
online lessons and in GitHub. Workshop materials will continue to be
open access and flexible, thus they can be readily adopted by
others. Adapting workshop materials is low cost and does not require
special equipment. Workshop materials are structured such that they
can scale to the size and application of interest to a particular
group. Anyone using workshop materials can directly contribute
changes and feedback, which both increases buy-in and improves the
materials organically. The Software Carpentry infrastructure
provides support in the form of materials and people. And finally,
local chapters of The Hacker Within create a natural ecosystem of
support for workshop participants, their peers, and faculty.

As a subordinate part of their work, the assessment researcher
employed by this project will assess the extent to which curriculum
developed during this program is taken up by other educators
(particularly those who think of themselves as scientists first and
computationalists second), and their perception of its
utility. Mid-point results of this evaluation will be shared with
the instructional designer in order to allow evidence-based
improvement of the materials.

Project Management

Prof. Slaybaugh will be responsible for overall project management
and reporting. The educational assessment expert hired by this
project will report directly to her. Prof. Slaybaugh will be
assisted by Dr. Huff, who will manage and coordinate the graduate
student assistants at each site. Dr. Huff will also be responsible
for organizing the workshops aimed at female students.

Profs. Barba, Brown, White, and P. Wilson will be responsible for
coordinating workshops and for recruiting and supervising the
graduate student assistant at their respective
institutions. Prof. Teal will co-coordinate the workshops held at
Michigan State University and will assist in developing workshop
materials.

Dr. G. Wilson and the half-time instructional designer hired by this
project will be responsible for preparation and publication of
learning materials. Dr. G. Wilson will also provide instructional
training for the graduate student assistants and other participants
in the project on an ongoing basis and will be responsible for
organizing the workshops aimed at students from minority serving
institutions.

Workshop operations (such as finding instructors and arranging their
travel) will be handled by Mozilla staff who are performing these
duties for the Software Carpentry program more generally. These
staff will be supervised by Ms. Thaney, who will also be responsible
for connecting the other PIs and the graduate student assistants
with other open and web-enabled science groups.

Related Work

Theoretical Positioning

Our theory of action is straightforward: if students are explicitly
taught software skills in a way that makes them seem both useful and
important, they are likely to begin using them in day-to-day work,
which will create a positive feedback cycle leading them to acquire
more (and more advanced) skills on their own. This positive feedback
cycle will in turn result in the students being more likely to
engage in open and web-enabled scientific activities that would
otherwise have been unknown, incomprehensible, or out of reach.

Using the terminology of, our work is primarily design and
development research. We plan to design and develop solutions
related to student engagement and mastery of specific skills,
drawing on existing evidence from Software Carpentry's
graduate-level workshops and investigating their impact and
effectiveness. Further, we further plan to design and iteratively
develop interventions. We are ready to begin collecting data on the
feasibility of implementing solutions in typical delivery settings.

Research

Studies of how scientists use computers and the web have found that
most scientists learn what they know about developing software and
using computers and the web in their research haphazardly and
through word of mouth. In our experience, most training meant to
address this issue:

does not target scientists' specific needs (e.g., is a general
"Introduction to Computing" class shared with students majoring in
other areas);

only covers the mechanics of programming in a particular language
rather than giving a complete picture, including data management,
web-enabled publishing, the "defense in depth" approach to
correctness discussed in, or the other foundational skills laid
out in; and/or

jumps to advanced topics such as parallel computing before
scientists have mastered the foundations. Most research on
scientific computing, such as, does the same.

On the other hand, studies of how people in general learn to
program, and of how effective different approaches to teaching them
are, have made significant strides in the past decade. In
particular, our work is informed by the long-running research
program of Guzdial et al. at Georgia Tech, who have found that a
"media first" introduction to computing outperforms more
conventional alternatives and that it is possible to assess the
extent to which programming concepts, rather than merely the syntax
of a particular programming language, have been mastered.

Others have demonstrated that peer instruction is a significantly
better way to teach introductory programming than conventional
classroom approaches. As discussed in the section below, we are
already working to incorporate these evidence-based approaches into
our teaching and will accelerate these efforts within the scope of
this award.

Software Carpentry

Software Carpentry is the largest effort to date to address issues
surrounding inadequate software carpentry skill training for
students. Originally created as a training program at Los Alamos
National Laboratory in the late 1990s, it is now part of the Mozilla
Science Lab's efforts to help scientists take advantage of ways in
which the web can change the practice of science today and invent
new ways tomorrow. Over 100 certified volunteer instructors
delivered two-day intensive workshops like those described earlier
to more than 4200 people in 2013 alone.

Software Carpentry's curriculum and teaching practices have been
refined via iterative design and are informed by current research on
teaching and learning best practices. Its instructor-training
program, which takes 2–4 hours/week of a trainee's time for
12–14 weeks (depending on scheduling interruptions),
introduces participants to a variety of modern teaching techniques
(e.g., peer instruction, active learning, and understanding by
design), to concepts underlying these techniques (e.g., cognitive
load theory), and to topic-specific work by computing education
researchers (see,, and the first third of for overviews). One
example of how they translate theory into practice is their
insistence on live coding during teaching as a way of demonstrating
and transferring authentic practice to learners.

Evidence of instructor training and experience is based on a
Mozilla's Open Badge program. Further levels of expertise are
obtained through helping at a workshop, being an instructor, being a
lead instructor, and developing workshop materials. All of the
participants in this proposal are certified instructors and have
taught or will teach Software Carpentry workshops prior to the start
of this project. For example, Dr. Huff has already taught at 9
workshops and Dr. G. Wilson has served as an instructor at 35.

Past Assessment

Software Carpentry has been assessing learning outcomes and
retention since the beginning of its Sloan Foundation funding in
January 2012. The first round of assessment included both
qualitative and quantitative assessment by Dr. Jorge Aranda (then at
the University of Victoria) and Prof. Julie Libarkin (Michigan State
University).

Dr. Aranda surveyed and interviewed participants, observed a
workshop, and analyzed screencasts of participants working through a
programming assignment. The surveys and interviews were conducted
both pre- and post-workshop. The survey asked questions regarding
the software development habits of respondents, the tools they were
familiar with, their level of knowledge of five core Software
Carpentry topics (shell commands, Python, version control, SQL, and
testing concepts), and their challenges in using scientific
computing to answer their research questions.

According to both qualitative and quantitative data, Dr. Aranda
found significant increases in participants' understanding and use
of shell commands, version control tools, Python, and testing
techniques. Perhaps more importantly, participants reported better
proficiency with software tools; greater concern for issues of
provenance and code quality; better strategies to approach software
development; and new research questions that have become
accessible thanks to an increase in participants' software
development skills.

Students also took a quiz consisting of yes/no questions that were
purposefully chosen so that only about half of them could be
answered with the standard material in the workshop; the other half
was not covered by workshop instruction. Additionally, a
cross-cutting half could be answered with introductory familiarity
to the topic in question, while the other half would represent more
advanced levels of expertise.

The objectives for this were to assess whether participants not only
learned the workshop materials but were exploring the topics in
greater depth on their own, and to avoid ceiling effects in our
survey. Quiz performance improved across the board (by ~30%) for all
question categories.

Prof. Libarkin performed a more detailed assessment of participants
in a workshop held at Michigan State University, which was attended
remotely by students from the University of Texas at
Austin. Prof. Libarkin also collected qualitative and quantitative
data. Eighty-five percent of participants reported that they learned
what they hoped to learn, 81% changed their computational
understanding, and 96% said they would recommend the workshop to
others.

An attempt to scale this up in 2013 was set back by personnel
changes, but systematic follow-ups with past participants in
workshops have now been resumed, and we expect to be able to present
preliminary results by mid-2014.

Some excerpted survey questions from the discussed studies can be
found in the full proposal, as can excerpted interview questions
from Dr. Aranda's study. The studies mentioned here and the
additional data that will become available within the next six
months will serve as a starting point for this project's assessment
expert.

The Hacker Within

The Hacker Within (THW) was founded by graduate students, including
Dr. Huff and Prof. Slaybaugh, in nuclear engineering at the
University of Wisconsin – Madison to provide a forum for
sharing scientific computing skills and best practices with their
peers. As THW matured as a student organization, it attracted
students from many scientific disciplines and academic levels. THW
conducted bi-weekly seminars and developed a series of short courses
addressing the programming languages C++, Python, and Fortran; best
practices such as version control and test-driven code development;
and basic skills such as UNIX mobility. This curriculum was
delivered primarily as interactive short workshops on campuses and
during scientific conferences. Many previous founders of the Hacker
Within have since become instructors with Software Carpentry, and a
new generation of THW graduate students has begun to emerge in their
place. In 2013, new branches of THW were initiated at the University
of Melbourne and the University of California, Berkeley (under the
direction of Dr. Huff and Prof. Slaybaugh).

Condo of Condos

The "Condo of Condos" consortium, led by Clemson University and
including the University of Wisconsin – Madison and four other
campuses during its pilot phase, has recently been recommended for
funding by the National Science Foundation. The consortium's primary
task is to build a network of advanced cyberinfrastructure research
and education facilitators (ACI-REFs), with goals that include
increasing the diversity of researchers using advanced cyber
infrastructure on each campus and developing data science
practitioners. The Software Carpentry workshops being designed under
this proposal will serve those goals directly. As an investigator on
both that project and this proposal, Prof. P. Wilson will engage the
network of ACI-REFs to share this curriculum with both the initial
consortium institutions and any institutions who are able to join in
the future.

D-Lab

D-Lab, located at the University of California, Berkeley, has the
mission to create cross-disciplinary resources for high-level
training and support services for social science researchers
campus-wide. It does this by adaptively building new forms of shared
infrastructure, including consulting and workshops in research tools
and methods, and fostering discovery and connections with the pools
of expertise and offerings of Berkeley's departments and
professional schools. As a new research unit at Berkeley, D-lab has
been actively involved in teaching software tools. D-lab is engaged
in evaluating students' experiences with those trainings and
adapting subsequent trainings in response to their
evaluations. D-lab is an enthusiastic supporter of campus efforts to
broaden this approach to teaching.

D-lab will provide space, outreach, and assistance in locating and
supporting the assessment position outlined in this
proposal. Prof. Slaybaugh will engage with D-lab to capitalize on
their expertise at the intersection assessment and scientific
education.

Broader Impact

We believe this work will have significant impact in several areas
beyond directly improving the computational science skills of
workshop participants.

Enhance economic competitiveness. Computing is no longer
optional in any part of science: even scientists who don't think
of themselves as doing computational work rely on computers to
prepare papers, store data, and collaborate with colleagues. The
better their computing skills are, the better prepared they will
be to contribute to the research that underpins the nation's
economic competitiveness.

Improving STEM education for everyone, not just
participants. By creating and validating high-quality open
access teaching materials, and the methods used to deliver them,
this project will enable improvement in STEM education for
everyone, everywhere, not just for participating students and
participating institutions.

Improving STEM education tomorrow, not just today. As
noted in the introduction of this proposal, most of today's
efforts to transfer computational skills to STEM researchers and
connect them with 21st Century innovations in how science is done
are flying blind: there is effectively no feedback from long-term
impact to instructional action. By creating and validating such a
feedback loop—i.e., by showing scientists how to apply
science to their teaching—this project will demonstrate how
STEM education can be continuously improved.

Improve participation in STEM by women and underrepresented
minorities. The disproportionately low participation of women
and some minority groups in STEM is well documented, as is the
fact that computing is one of the least diverse fields within
STEM. This second fact creates a vicious cycle: people with weaker
computing skills may be less competitive in research than their
peers, which reduces their participation in activities viewed as
non-core, which in turn results in them having weaker skills. This
project will strive to break this cycle by giving at-risk students
an opportunity to "level up" in a supportive environment and by
connecting them with mentors who can serve as role models.

Career Management Plan

The graduate students who are serving as mentors for the
undergraduates at the different universities will each be paired
with a local faculty mentor. The faculty mentor will meet regularly
with the graduate student to discuss and problem solve any issues
that the graduate student or undergraduates are having and to
provide active mentoring on how to train students in computational
approaches.

In addition to engaging with the graduate students on their
mentoring of the undergraduates, the faculty mentors will also serve
as mentors for computational aspects of the graduate students'
research and careers. In many areas of science,
computationally-minded students are located in labs where the PIs do
not have strong computational backgrounds. This means that they do
not have a mentor to teach them about good computational practice in
research. In addition, they do not have someone with whom to discuss
computational careers, thus limiting their exposure to career paths
outside of academia. Because the faculty mentors will have strong
computational backgrounds themselves, they can fill this void for
computationally-minded students.