State University of New York: Course Redesign Initiative

Improving Learning and Reducing Costs:
A Summary of Program Outcomes

By Carol A. Twigg

From June 2007 to June 2010, the National Center for Academic Transformation (NCAT) conducted a program in course redesign in partnership with the State University of New York (SUNY) called the SUNY Course Redesign Initiative (SUNY CRI). The goals of this partnership were to 1) build on the successful models and lessons learned from NCAT’s national programs to create a course redesign program within SUNY; 2) adopt new ways to improve student learning outcomes and demonstrate these improvements through rigorous assessment; 3) reduce instructional costs and free up instructional resources for other purposes; 4) produce a number of successful SUNY course redesigns that can be spread to institutions throughout the system; and, 5) develop the internal capacity of SUNY faculty and staff to continue the course redesign process. All of those goals have been achieved.

The SUNY CRI awarded grants of $40,000 to support the redesign process through a competitive process. NCAT engaged the SUNY institutions in an extensive planning process to teach applicants its principles of course redesign. The NCAT process required faculty teams to follow a highly structured, iterative course redesign development process. Of the 21 teams which began the process, 10 were ultimately selected for funding. The final selection was made by SUNY Central Administration. The following proposals were funded: Buffalo State College: The Economic System; SUNY Canton: Introduction to Biology; SUNY Canton: Basic Communication*; Erie Community College: Preparation for College-Level Writing*, SUNY Fredonia: First-Year Spanish; Niagara County Community College: Introduction to Statistics; SUNY College at Old Westbury: College Algebra; SUNY at Oswego: College Algebra; SUNY at Potsdam: European and US History; and, Stony Brook University: Physics for Life Sciences. Eight of the 10 projects completed the redesign process and fully implemented their redesigns in fall 2009. (Those asterisked did not complete the process.)

The results achieved by the SUNY CRI were very strong. The redesigns impacted approximately 5,000 students. Four of the eight projects demonstrated improved student learning as measured by direct comparisons of content mastery. Two of the remaining four produced learning equivalent to the traditional format. The assessment results of the remaining two were unclear due to problems with the instrument (but each showed increased completion.) Five of the eight projects improved course completion rates (as measured by a final grade of C or better). One showed a completion rate equivalent to the traditional format. Completion rates in two projects declined. Seven of the eight projects reduced their instructional costs—on average by 35%, and two projects saved more than originally projected. (One project was not able to carry out its cost savings plan due to a state-imposed hiring freeze.) The annual savings for the eight projects was $534,655. All eight of the redesigns will definitely be sustained after the grant period is over.

NCAT’s recommendations to SUNY for future action included:  1) SUNY should find ways to publicize the results of the SUNY CRI among its colleges and universities; 2) SUNY should conduct a second round of the SUNY CRI; 3) SUNY should engage in a transition process from NCAT’s managing a course redesign initiative to doing so itself; 4) provosts’ offices on each campus should be more purposeful and more actively involved in generating course redesign project possibilities and should think about incorporating redesign as a factor in campus resource allocations; and, 5) SUNY should consider requiring that each campus integrate course redesign into its campus allocation strategies.

Project Outcomes

What follows is a summary of the SUNY course redesign projects’ final reports.

1) Did student learning improve (as measured by direct comparisons of content mastery)?

2) Did course completion rates improve (as measured by comparisons of final grades)?

3) Were instructional costs reduced?

4) Will the redesign be sustained after the grant period is over?

1) Did student learning improve (as measured by direct comparisons of content mastery)?

Yes

1. Buffalo State Economics  (compared common exam questions)

• Learning outcomes were assessed by comparing common exam items on the final exam. The percentage correct for the fall 2008 traditional section was 57.7%, for the fall 2008 partially redesigned section 67.9%, for the spring 2009 partially redesigned section 60.6%, and for the fall 2009 fully redesigned section 66.3%.

• A one-way analysis of variance comparing the percentage correct of the available common items across all four sections showed a significant difference (p < .001).  Post hoc analysis showed that the spring 2009 mean percentage was not significantly different than the traditional class from fall 2008; both were significantly lower than the fall 2008 partially-redesigned course and the fall 2009 fully redesigned course.

2. Canton Biology (compared common pre-tests and post-tests)

• Students in the fall 2009 redesigned course (N = 244) gained 18 points on pre/post-test performance compared with students in the spring 2009 traditional course (N = 78) who gained 12 points.

3. Niagara County Community College Statistics (compared common final exams)

• The average final examination score increased from 64.9% for the traditional course to 68.7% for full implementation of the redesigned course. This difference is statistically significant at the 95% confidence level.

4. Potsdam History  (compared common multiple choice questions and essay exams with common rubrics)

• In the American History course, average scores on comparable essay questions, graded by the same rubric, improved from 2.22 in the traditional course to 2.58 in the redesigned course. Correct responses to common multiple-choice questions increased from 55% to 76%.

• Averaging factual (multiple choice) scores with interpretive and analytical (essay) scores in European History yielded a measure of overall learning that compares very favorably with the traditional course.  Scores at the top of the distribution more than doubled (from 5% to 11%), while those in the “B” range rose significantly (from 39% to 46%).  “C”-range performances diminished substantially (from 27% to 18%) while “D” scores plummeted (from 23% to 4%).

No difference

1. Old Westbury College Algebra (compared common final exams)

• When comparing results of the three semesters (fall 2008, spring 2009 and fall 2009) when both traditional (N = 525 students) and redesigned sections (N = 516 students) were offered, there was no significant difference between the traditional and redesigned sections in final exam performance.
  

2. Oswego College Algebra (compared common pre-tests and post-tests)

• A comparison of pre/post-test scores showed a slight decrease in the points gained in the fall 2008 traditional format from 38 to 36 in the fall 2009 redesign full implementation. The 2009 cohort began the course with a weaker background. The average pre-test score was 13 for the fall 2009 students compared with 16 for the fall 2008 students. This difference was not statistically significant.

Unclear

1. Fredonia Spanish (compared common final exams)

• On the reading assessment, traditional students outperformed redesign students: 81% of traditional students scored 16 or more points out of a possible 20; 67% of redesign students scored 13 or more points.

• On the listening assessment, traditional students outperformed redesign students: 77% of traditional students scored 13 or more points out of a possible 15; 51% of redesign students scored 13 or more points.

• On the writing assessment, redesign students outperformed traditional students: 56% of traditional students scored 15 or more points out of a possible 20; 65% of redesign students scored 15 or more points. After discussing the grading of the writing component, the team decided that there were discrepancies in the application of grading criteria despite the use of a common rubric, which calls into question these results.

• The team believes that the assessment tool was inadequate in that it addressed a more traditional way of learning rather than the innovations that the redesigned course included such as more communication/conversation during class time. The instructors believe that students from the redesigned course were more prepared than was demonstrated by the assessment scores. They have communicated more in class, and they performed well on oral tests.

• This perception is supported by the fact that the percentage of students receiving a grade of C or better in the traditional course was 84%, and the percentage of students receiving a grade of C or better in the online portion of the redesigned course was similar (83%). The percentage of students receiving a grade of C or better in the in-class portion of the redesigned course (communication/ conversation), however, was 93%.

2. Stony Brook Physics (compared common pre-tests and post-tests)

• The improvement of the test scores in the traditional course was from 24.3% in the pre-test to 42.0% in the post-test. The improvement of the test scores in the redesigned course was from 29.0% in the pre-test to 54.6% in the post-test.

• Although serious efforts were made to have uniform test conditions in both course types, there were differences in the execution of the post-tests. The redesigned course instructor allowed students to use a student-written formula sheet and graphing calculators whereas the traditional course instructor did not. (The traditional course was taught by a faculty member who was not a member of the redesign team.) The team concluded that students in the redesigned course, although showing an apparent larger pre/post–test improvement may not have done better than students in the traditional course in view of the difference in post–test execution.

2) Did course completion rates improve (measured by comparing final grades)?

Yes

1. Buffalo State Economics

• In fall 2008, the traditional lecture section showed a success rate (grades of A, B or C) of 67%.  This was significantly lower (chi square, p < .002) than the success rate in the partially-redesigned section in the fall 2008 pilot, which was nearly 85%. 

• Subsequent semesters showed a success rate in between those two extremes: a 76% success rate for the spring 2009 partial redesign, and a 79% success rate for the fall 2009 fully redesigned course.

2. Fredonia Spanish

• The percentage of students receiving a grade of C or better rate in the traditional course was 84%. The percentage of students receiving a grade of C or better rate in the online portion of the redesigned course was 83%. In the in-class portion of the redesigned course, the percentage was 93%.

3. Old Westbury College Algebra

• The passing rate (grade of C or better) for the fall 2009 redesigned sections was 70.4% compared with the fall 2008 traditional sections passing rate, which was 69.3%.

• The passing rate for the fall 2009 redesigned sections was significantly higher (99% confidence) than the average fall semester passing rate for 2003 – 2007, which was 62%.

4. Oswego College Algebra

• The percentage of students earning a grade of C or better increased from 42% in the traditional course to 52% in the redesign.
  
  
• There was a significant reduction in the DFW rate,  29.5% for the fall 2009 redesign as opposed to the fall average rate from 2003-2007 of 37.5%. The difference was significant.

5. Stony Brook Physics

• The percentage of students completing the course successfully (a grade C or better) was 83% in the traditional and 86% in the redesigned course.

• The percentage of those receiving a grade D or better was 91% and 95% for the traditional and redesigned course respectively.

No difference

1. Canton Biology

• During the spring 2009 redesign pilot implementation, the percentage of students earning a final grade of C or better was 48% compared with 43% in the spring 2009 traditional course.
  
  
• During the fall 2009 full implementation of the redesign, the percentage of students earning a final grade of C or better was 50% compared with 53% in the fall 2008 traditional course. Completion rates in this course are high, averaging around 93%.

No

1. Niagara County Community College Statistics

• The redesigned course experienced a decrease in successful course completion rates (grades of C or better) from 62.4% for the traditional course to 58.3% for full implementation of the redesigned course.
• The DFW rate in the redesigned course was 30.6% compared to a DFW rate of 22.8% in the traditional course. 

• A contributing factor to these results is that a portion of students who passed the final exam did not complete the required assignments. Such students may have been successful in the traditional course where homework completion did not contribute to their final grade.

2. Potsdam History

• Student success rates (grades of C or better) declined in both the American History and European History. In American History, 73% of traditional students received a grade of C or better compared with 61% of redesign students. In European History, 75% of traditional students received a grade of C or better compared with 63% of redesign students.

• It may be that decreased completion is the paradoxical result of the course redesign’s pursuit of other goals to make the department’s instruction in history surveys more uniform and to foster greater student engagement.  Since generally less demanding adjunct faculty have been eliminated from the American and European survey courses and grading has become more uniform, it may be that past grades were higher because the grading was easier. Because the redesigned courses require a greater degree of weekly engagement by every student than traditional courses formats, it may be that less motivated students wash out more readily.

3) Were instructional costs reduced?

Yes

1. Buffalo State Economics

• The redesigned course reduced the cost of instruction by reducing the number of sections from two per semester to one, and increasing section size from ~ 120 to ~240 students.

• The number of full-time faculty teaching the course decreased from two to one each semester.

• These actions decreased the cost-per-student from $94 to $51, a savings of 46%.

2. Fredonia Spanish

• The projected cost savings of more than 40% for the students who needed two semesters was achieved as planned. One intensive semester fulfilled the requirement usually completed in two semesters. Therefore, the following semester all the resources (instructors, classrooms) were committed to serve an entirely new group of students.

• The fact that most students were able to take this high demand course was very important for the department and the university. In the past, many students have had problems finding a seat in the course, and sometimes this has set back their academic plans and jeopardized their graduation plans.

3. Niagara County Community College Statistics

• The redesigned course adhered to the cost savings plan. The redesigned course reduced instructional costs from $339 per student to $162 per student, a 52% savings.

• Cost reduction strategies included reducing the number of sections covered by full-time faculty from eight to five and increasing the number of sections covered by adjunct faculty from two to five. Each full-time faculty member was partnered with an adjunct faculty member to team-teach two sections simultaneously.

4. Oswego College Algebra

• The redesign achieved greater savings than anticipated. The cost of the traditional course was $77,400; the cost of the redesigned course was $37,400. This represents a 52% savings, compared to a planned 42% savings.

• These savings were achieved, as planned, by changing the mix of personnel and reducing the number of instructors from seven to two, instead of seven to three as planned.

• Traditionally, nine sections of ~25 students (N = ~200 students) were offered each year. The original redesign plan envisioned six sections of 50 to reduce the number of sections and accommodate an anticipated increase in student enrollment to 300, which has not yet occurred.  As a result of the pilot and full implementation semesters, the team restructured the course to include three sections of ~75 annually (N = ~225 students.)

• In addition, beginning in spring 2010, the number of required lab meetings was reduced from three to two per week, resulting in an additional cost savings from undergraduate learning assistant (ULA) salaries.

5. Potsdam History

• The redesign plan projected a cost savings of between 20% and 25% for the new courses. The cost savings plan was followed as proposed. The redesigned courses did not realize their potential to reduce instructional costs during the 2009-2010 academic year because enrollments ran well below capacity. Enrollment trends suggest that this problem will be quickly overcome. Between fall 2009 and spring 2010 semesters, enrollments in the European course increased by nearly 60%. In the American course, they increased by over 10%. The department expects enrollments much nearer to capacity in the 2010-11 academic year, which will yield real savings.

• The redesign has also allowed the department to make real progress on a college-wide goal of reducing faculty teaching loads from 24 to 21 hours per year.

6. Old Westbury College Algebra

• The cost savings plan, which projected a reduction in the cost-per-student from $176 to $155, was carried out as planned.

• In addition, the plan estimated $18,250 for lab costs for the full implementation of 300 students in fall 2009, approximately $61 per student. The redesign course utilized the same lab as another math course, and the cost to operate that lab for the fall 2009 semester totaled $27,800. Since both courses used the same lab, there were 500 students utilizing these services as opposed to the 300 College Algebra students in the plan. Therefore, the cost-per-student for the lab was actually $56 instead of $61, producing an additional savings of $5 per student.

7. Stony Brook Physics

• The team implemented the originally planned cost savings. The overall cost-per-student was reduced from $463 to $236, a saving of 49%.

• The elimination of faculty-taught recitations in particular produced the bulk of the savings and liberated faculty for an expansion of help rooms for more courses.

• Changes in the format of the lab made possible a 20% increase in the number of students per graduate teaching assistant.

No

1. Canton Biology

• The plan to increase section size from 60 to 80 and reduce the number of sections offered from eight to five was carried out.  One of the primary cost saving measures of the redesign was not implemented: the hiring of a course coordinator (CC). The person appointed to this position left the college for another job two weeks before the start of the semester. The state imposed a hiring freeze at about the same time; therefore, the team was unable to refill the position. The tasks of the course coordinator fell to the lead instructor.  While this increased the planned work load of the lead instructor, the load was equivalent to that of teaching the traditional course. The replacement of one lecture period a week with online learning activities and the resulting decrease in hours needed to prepare for class offset the loss of the course coordinator.

4) Will the redesign be sustained after the grant period is over?

Yes

1. Buffalo State Economics

• The economics department has been pleased with the results of the redesign: cost savings that have allowed the reallocation of faculty resources to the graduate program, and a student learning experience that has equaled or surpassed the past approaches to the course. Therefore, the department is committed to the redesign approach.

2. Canton Biology

• The team is steadfast in its belief in and its intention to carry on the redesign in its current form. The current economic climate in New York State higher education is dictating higher enrollments and larger class sizes taught by fewer faculty members. The redesign has fulfilled both of these criteria. After two semesters of teaching the redesigned format, the team has no thought of going back to the traditional lecture format.

3. Fredonia Spanish

• Now that the full implementation has been completed without any extraordinary problems, it will be easy to continue the course.

• The fact that most students were able to take this high demand course was very important for the department and the university. In the past, many students have had problems finding a seat in the course, and sometimes this has set back their academic plans and jeopardized their graduation plans.

4. Niagara County Community College Statistics

• Without question, NCCC will sustain the changes that were undertaken through the redesign. In light of the current economic climate, the cost savings produced in the redesigned course were significant.

• Therefore, the redesign format is being used as a model for other courses and future redesigns in the mathematics department. The redesign approach is currently being extended to other mathematics courses where appropriate. The NCCC administration is fully supportive and encourages the creation of redesigned courses.

5. Old Westbury College Algebra

• The sustainability of the project is not in question at this time. Support from administration has been extraordinary with funding being maintained even when budgets have been severely cut throughout the state.

6. Oswego College Algebra

• College Algebra will continue to be offered in the redesigned format for the foreseeable future.  The cost savings achieved from increasing section size from 25 to 75, the increase in quality achieving by standardizing course content and positive student evaluations are compelling reasons to continue the initiative.

• In recent years, with the declining number of full-time tenure track faculty, it has become more of a challenge to offer the requisite number of courses.  As the redesign continues to be refined, the team anticipates the possible extension of this model to other appropriate courses.

7. Potsdam History

• The redesigned history courses enjoy departmental as well as college-wide support and the commitment of relevant faculty to teach these classes in the foreseeable future. Additional faculty members are preparing to assume responsibility for the redesigned courses, thus providing a deeper bench of instructors.

• In addition, the course redesign fueled interest beyond the department. SUNY Potsdam’s School of Education has solicited the history department to prepare courses in American and European history, redesigned on the same hybrid model, that would be specially tailored to New York State teacher certification standards.

8. Stony Brook Physics

• The redesign was a combined effort of the physics department, university administration and IT professionals and will continue with modifications. From fall 2010 onwards, the smaller courses (Physics for Life Sciences I and II) with ~ 200 students (which begin in the spring semester and end in the fall semester) will be redesigned as well.

Final reports from each completed redesign are available at http://www.thencat.org/States/NY/SUNY Project Descriptions.htm. Final reports include learning outcome data, course completion data, cost reduction data, a discussion of the most important pedagogical techniques that led to increased learning, a discussion of the most important cost reduction techniques that led to reduced costs, a discussion of implementation issues encountered during the redesign process, and a discussion of future sustainability of the redesign.

Quick Links:

State University of New York Main Page..