6
Background on “What Works” Analysis: Studies of CTE focused on enhancing teacher knowledge and practice Links to student learning Method: –Examined “standards” set by AFT, NSDC, NCTM, scholars –Examine published/reviewed studies that link professional development to student achievement –Allow in studies where measure of teacher learning is linked to student achievement –Descriptive studies of what’s “in” the professional development widely available to teachers

19
What Works? Content What works –Focus on the content –Focus on student learning –Active learning –Examples from classroom practice What doesn’t work –Focus on the content –Focus on student learning –Active learning –Examples from classroom practice What doesn’t work –Focus on the content –Focus on student learning –Active learning –Examples from classroom practice Success Depends Upon Enactment

20
“Motion Math” K-6 workshop Meets several “standards” for best practice in PD –Teachers explored activities they could later do with students (practice-focused) –Worked collaboratively –Focus on mathematics and student learning

21
Motion Math: Space Alien Math “Motion Math believes in a hands-on approach to operations. Our leader recently became interested in proportionality, and discovered a new thing using it: every division problem has four numbers. Every division question is the same as saying “how much in one?” It becomes a set of proportional fractions when you think about it this way. One can cover any portion of the square and it becomes alternately a division or multiplication problem.” We didn’t slow down to explain which would be which. Several teachers seemed confused. Later, she clarified: “Division is really a ratio. All we are doing is fair sharing.” 1

22
Motion Math: Space Alien Math Eighteen aliens are on the moon. They use two spaceships to come back to earth. How many will be in each spaceship?

23
Motion Math: Meeting the “Focus on Mathematics” Standard? DOES connect division and proportionality, potentially rich However: Flawed presentation, mathematically –Superficial, disconnected treatment: Only a brief introduction – room for confusion ( “ Every division question is the same as saying ‘how much in one?’”) Not clear how this relates to more standard interpretations of division (“How many groups of 2 in 18?”) Not clear how the aliens are related to the symbolic division; where is the “1”? Confusing follow-up: “Division is really a ratio. All we are doing is fair sharing.” Flawed problem!!!! 1

24
Example II: Writing Mathematics Focused on helping teachers help students build mathematical explanations for state test Met many standards, in name –Collaborative –Teachers did mathematics problems –Used artifacts of student work –Follow-up to initial PD session

26
Writing Mathematics: Meeting the “Student Learning” Standard Graded student work on “open-ended” problems –Correctly identified student problem with regrouping –But remedy is purely procedural: [The leader] had a little colloquy with a teacher about learning place value and the idea of trading. He said when he goes into a classroom, he gets kids to make a list of “sad” (7,17,5,6) and “happy” (13,2,22,10) numbers in reference to the number 23. The teacher at my table immediately recognized the sad numbers as those you have to regroup, happy numbers as the ones which you didn’t. Leader said that every student should have a sense for that, if you’re working with 5, for instance, you have to regroup numbers that end in 4,3,2,1 and 0, that you don’t have to with 6,7,8,9.

28
Student Mathematics (Teachers discuss how to solve one problem, , using multiple methods). Then the provider showed another video, this one with a student solving an addition problem in which neither number was close to 100. The student narrated the problem (24+37) like this: “Twenty- four plus thirty is fifty-four. Fifty four plus six is seventy, plus one is seventy one.” The provider asked teachers to talk to others around them, explaining what this student had to know to do this problem in this way. After members of the audience did so, the provider explained this student’s thought process, saying that he “holds that seven out,” then breaks it into two pieces to add. She challenged teachers to determine whether this student followed the rules and whether the method this student used worked only in a few cases or generally for all two-digit addition problems.

29
Take-Home It’s enactment that matters –Is there “academic press” – in terms of covering challenging mathematical content? –Is the professional development “mathematically dense”? –Is there rich, meaningful mathematics? –Are substantial connections made to how students will use/learn material?

30
Things that People Think Work But For Which There’s No Rigorous Evidence

33
Unsolicited Advice Learn, learn, learn –Morally obligated to find out whether your professional development works –With your help, we might know something in 10 years –Rigorous evaluations needed Not only in sense of good design, but in careful implementation (i.e., no lost teacher IDs), careful instrumentation (i.e., no homegrown measures with ?? reliability and validity), and thoughtful links to student achievement

34
Unsolicited Advice Rigorous evaluations –Mechanism for making causal inference: What would have happened to these teachers’ teaching and learning in the absence of this program? –Measurement instruments that are objective and credible –Longitudinal measurement –No logistics problems to sink ships –Strong analyses, with plausible story about student achievement change (value-added estimates; controls for student SES, ELL, HLM)

35
Bargain-Basement Evaluations (Or, why I will be hunted by the IES police) Use teacher as unit of analysis N=50 (25 treatment, 25 control) Pre/post test teachers as condition of participation in the learning experience Pre/post test students in those teachers’ classes as condition of participation –Use established test close to your own curriculum –Administer in fall/spring at time point that makes sense Use HLM as analytic method (kids nested within teachers) –Treatment variable is at teacher level –Now available in “mixed models” in SPSS

39
Middle school teacher MKT, predicted by credentials and experiences Former high school teacher.24** High school credential.66**** Mathematics credential.39**** Years experience.008* Mathematics classes.17*** Math methods classes.08* Adjusted r-squared.32 Regression MKT as dependent “Omitted” category is middle school credential only Strong evidence that high school credential/ experience associated with high MKT