Tuesday, January 31, 2017

This NSF study “proves”
that active learning is better than just lecturing.

The devil is in
the details.

The quote from the
sited “study”: “Some 225 of those studies met the standards to be included in
the analysis including: assurances the groups of students being compared were
equally qualified and able, instructors or groups of instructors were the same,
and exams given to measure performance were either exactly alike or used
questions pulled from the same pool of questions each time.”

And another
quote:

“Freeman and his
co-authors based their findings on 225 studies of undergraduate education
across all of the "STEM" areas: science, technology, engineering and
mathematics.”

To meet the
requirements of an actual scientific study all “sub-studies” must had used THE
same (or convertible) measuring instruments and procedures.

Clearly, measuring
tools and procedures depend on the teaching subject. However, that is not an
actual issue.

There is no way
to follow the logic which lead from data to conclusions (for example, in math sciences,
if one proves a theorem, everyone else can trace all the steps, just in case –
for a human factor – if a mistake was made – that IS science).

The most important
issue with this type of meta-analysis is that it based on the data which initially
do not fulfill the requirements of a scientific study.

We read that “those
studies met the standards to be included in the analysis including: assurances
the groups of students being compared were equally qualified and able,
instructors or groups of instructors were the same, and exams given to measure
performance were either exactly alike or used questions pulled from the same
pool of questions each time.”

This statement
is very misleading.

When we read it,
it sounds like all study in physics were based on using the same measuring instrument
and procures; or all studies in each another subject area also were based on
using the same measuring instrument and procures.

If that was a
case that would be grate!

In reality, it
only means that authors of one paper on effects of active learning in physics
used the same (or similar) exams when testing studies in a controlled group and
in the experimental group. But other researchers in a different study were
using exams (or other assessing techniques) different and incomparable with
ones used in other studies.

However, if we examine
each individual study we find that they do not sustain the same scrutiny we
would apply to a study in physics, mathematics, or any other STEM field.

If all
individual studies do not satisfy criteria for a scientific study, and if for
all individual studies the results can be explained by other reasons than
active learning, the conclusions of the meta-analysis have no solid basis.

It does not mean
that I state that active learning does not work. I only say that we have no
solid evidence for making the statement that active learning is better than a
lecture.

And there are
also much simpler explanations for an effect provided by active forms of
teaching, for example, “the first law of TeachOlogy”: “If we take two large
groups of similar students, and one group of students will have a more extensive
or diverse learning experience (for example, more contact hours, or more time
spent on certain exercises, or training through more and different exercises,
etc.) students from that group, on average, will demonstrate better learning
outcomes than student in the controlled group.” (more at http://www.teachology.xyz/WNSF.html
and http://www.teachology.xyz/6LT.html).