 |
 |
|||||||||||
 |
||||||||||||
 |
How to Redesign a Developmental Math Program Using the Emporium Model VIII. How to Calculate Comparative Completion Rates Completion rates refers to the percentages of students who began the course and finished with a grade of C or better. This measure (sometimes referred to as a pass rate) is generally accepted in higher education to indicate student “success” in a course. Completion rates are not the same as measures of student learning. Assessment of learning refers to direct and comparable measures of student learning outcomes; completion rates refers to final grades. Q: Why are grades not comparative measures of student learning? A: Pass rates (grades of C or better) in traditional courses are not reliable indicators of student learning and are almost universally inflated due to prior inconsistencies in grading practices. Students in traditional courses are assessed in a variety of ways that lead to overall grading differences. Inconsistencies include (1) curving, (2) failing to establish common standards for topic coverage (in some sections, entire topics are not covered, yet students pass), (3) having no clear guidelines regarding the award of partial credit, (4) allowing students to fail the final exam yet still pass the course, and (5) failing to provide training and oversight of instructors, especially part-time ones. Q: Why would one want to look at comparative completion rates as well as comparative measures of student learning? A: It is important for students to both master the content of the developmental math sequence and complete the sequence as rapidly as possible in order to enroll in college-level courses. It is possible to demonstrate increased student learning through redesign (e.g., final exam means that increase from 50 percent to 70 percent), but if only 5 percent of students take the final exam, you have a problem despite the demonstrated increase in student learning outcomes. Q: Why are course-by-course completion rates not true measures of success in the Emporium Model? A: Ideally, one wants to see an increase in both student learning outcomes and completion rates. Unfortunately, there is often a discrepancy. In conducting an extended analysis of situations in which a modularized Emporium Model produced increased learning outcomes and decreased course completion rates, NCAT discovered a variety of reasons that course-by-course completion comparisons are not true measures of the success or lack of success of the model. Among them are:
In the traditional format, students typically exit the course by simply attaining a total cumulative score of at least 70 percent or 75 percent. Based on the averaging of grades, students can earn a C or better by passing enough tests and learning enough competencies but not necessarily all. In traditional sections, students often continue on to the next topic without having demonstrated mastery of the previous one. Increasing the mastery level above 70–75 percent to 80–90 percent, as many redesigned courses do, essentially raises the cut score for a student to earn at least a C in the redesigned course. When one uses a mastery learning approach, students do more work and learn more, which often takes longer. That means that many students do not complete a particular course by the end of the term. They can, however, start in the subsequent term where they left off in a previous one. Mastery learning, while sometimes taking longer to accomplish, ensures that students are well prepared to take on college-level work. Q: If course-by-course comparisons are not valid, how can we measure completion in the Emporium Model? A: Having recognized the difficulty in using course-by-course completion rates to compare student success, NCAT recommends two valid ways to measure student completion other than course-by-course comparisons. Making-progress grade. NCAT recommends that institutions award a making-progress (MP) grade to students who are making substantial progress at a high mastery level but have not yet completed the course or the course equivalent by the end of a given term. Definitions of MP grades should be roughly equivalent to a grade of C or better in the traditional courses (e.g., must have completed 86 percent of modules at 80 percent mastery, 80 percent of modules at 70 percent mastery, 75 percent of modules at 75 percent mastery, 75 percent of modules at 80 percent mastery). The Completion Forms included in Appendix B list an MP grade along with a place to indicate how the grade is defined. As an example, among Changing the Equation institutions (described in the Introduction) that were able to calculate a course-by-course completion rate, the success rate (grade of C or better) was 33 percent. After adding the MP grade to the calculations, the percentage rose to 74 percent. The latter is a much more valid indicator of the success of the program. Completion of the developmental math sequence. To evaluate the success of the Emporium Model in developmental math, you can compare the rate of completion of the developmental math sequence. To do so, you need to create two cohorts of students (one of students enrolled in the traditional course in the past and one of students enrolled in the redesigned course) and track the progress of both. If your traditional developmental math sequence comprised two courses, calculate the percentage of both cohorts of students who completed the sequence (receiving a C or better) in two terms. If your traditional developmental math sequence comprised three courses, calculate the percentage of both cohorts of students who completed the sequence in three terms. These calculations will produce a valid comparison of student completion rates. Q: What about measuring subsequent success in college-level math courses? A: Another way to evaluate the success of the Emporium Model in developmental math is to compare student rates of success in subsequent college-level math courses. This measure also requires you to create two cohorts of students (one of students enrolled in the traditional course in the past and one of students enrolled in the redesigned course). You then calculate the percentage of both cohorts of students who completed the subsequent college-level course(s) by receiving a C or better. Examples
To truly evaluate the success of the Emporium Model in developmental math, the rate of completion of the developmental math sequence and the rate of success in subsequent college-level math courses are the two most important data points to use. If only 20 percent of students exit the developmental math sequence but 75 percent pass the college-level course, you still have a problem, just as you did when 50 percent exited the sequence but were unprepared and only 30 percent passed the college-level course.
|
 |
Introduction Appendices: |
 |
||||||||
 |
|
|
|
|
|
|
|
|
|
|
|
|