Printable Version 

2004 NSF K-12 Math, Science, and Technology Curriculum Developers Conference

Math Workshop - Research and Evaluation

Paul Goldenberg,
Eric Karnowski,
Nina Shteingold

In the poster, we’ll describe the design of research and evaluation part of the Math Workshop project and discuss some implementation issues that are emerging from the formative research. Math Workshop Research and Evaluation

Research is an integral and extensive part of the development of the MW program. “Formative” research, conducted during the development of the project, guides the drafting and revising of program materials. “Summative” evaluation of the curriculum, to be conducted in classrooms using the final version of the materials, will address the effectiveness of the program in terms of student outcomes and teachers’ ideas about mathematics and classroom pedagogy.

Formative Research
There are two phases of formative research—pilot and field testing. Math Workshop materials will undergo two revisions: one after the pilot year based on feedback from the pilot testing data; and a second revision, based on the analysis of field test data. During both phases, we attend to students’ and teachers’ reactions, but the first (pilot) phase ensures that the lessons themselves are appropriate in structure and content, while the second (field-test) phase ensures that the materials can be handled by a wide variety of teachers and in a wide variety of school situations.

Pilot Testing
Pilot testing is conducted in a small number of local classrooms, to permit close monitoring of lessons through regular contact with teachers and students. Data collected during the pilot testing provide information about the usability of the lessons, challenges (and successes) that teachers experience in teaching lessons, questions teachers have about lessons and/or needs they express, students’ difficulties (and successes) with particular aspects of lessons, student engagement with the materials, and the overall coherence of lesson structure.

Field Testing
Field testing is designed to provide data about usability and effectiveness of the MW materials under more “typical” teaching conditions and in a greater variety of classroom settings. Data from field testing serve two purposes: they inform the final revisions of the materials (expected to be of the “fine-tuning” type and to involve mostly simplifications, clarifications, and, perhaps, occasional interpolated lessons or practice, all aimed at ensuring that the materials are maximally usable by a broad range of teachers) and they provide evidence of student and teacher learning.

Summative Evaluation
The summative evaluation will be conducted by an outside evaluator. It will use an experimental (randomized) design and will collect data for 10 MW and 10 comparison classrooms each at grades 2 and 4. These 40 classrooms will be drawn from 5 districts, some of which will also be participating in the field test. These districts will be selected to provide geographic, economic, ethnic, and academic student diversity.

This project is supported by the National Science Foundation (Grant No. ESI-0352345). Any opinion, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.