 |
 |
|||||||||||
 |
||||||||||||
 |
HOW TO ORGANIZE A CAMPUS-WIDE COURSE REDESIGN PROGRAM USING NCAT'S METHODOLOGY IX. Monitoring the Redesign Implementations Implementing a course redesign involves four phases: planning and development, conducting a pilot term, revising the redesign plan as needed based on the pilot experience, and fully implementing the redesign in all sections of the course, including assessing and evaluating the full implementation. The purposes of this part of the program are to implement the sound plans that have been developed and to follow through so as to ensure that adjustments get made where needed, roadblocks get overcome, and models of successful redesigns get achieved. During the redesign implementation process, it is critical that the program leaders monitor the redesign teams’ adherence to the teams’ proposals as a way of making sure the teams are actively following their plans for both quality improvement and cost reduction. Based on NCAT’s experience with more than 200 large-scale course redesign projects, we know that because of unanticipated issues that can arise, projects can get derailed during the implementation phase. Teams may not know how to respond, and their initial reaction is often to revert to the status quo of the traditional model. At such junctures, teams need to seek advice from experts in course redesign, who can discuss the problems with the teams and offer strategies for resolution. If changes get made that have an impact on either cost reduction or quality improvement, the program leaders need to discuss the implications with the teams and suggest alternative strategies. We cannot overstate how frequent monitoring and active intervention during the program implementation period greatly increase the likelihood of success. Throughout the program implementation period, the program leaders should monitor each phase and actively consult with the teams as appropriate. Informal but consistent ongoing progress reporting is important to make sure projects stay on track. In between formal reporting dates, teams should be required to submit regular progress reports to the program leaders via email or face-to-face meetings. The program leaders should review the redesign teams’ work and offer suggestions for improvement. Monitor the Planning and Development Phase During the six months prior to the pilot term, redesign teams engage in concrete preparation for the pilot. Teams meet and make necessary changes to course content or other aspects of the expected student experience such as modifications to classroom or lab space, to the design of web materials or other student guides, to the planning for student and faculty training, and to data-gathering preparations for effective assessment. And they perform other such activities that must be completed in advance of the pilot term. Monitor the Pilot Implementation During the spring term, teams pilot their redesigns with subsets of students and include all or almost all aspects of the redesigns. NCAT recommends that every large-scale redesign conduct a pilot before moving to full implementation. What do we mean by a pilot? A pilot involves testing the redesign idea—including most if not all of the important quality improvement and cost-saving characteristics of the planned redesign—with a subset of students enrolled in the course. Enrollment in the pilot section(s) needs to be large enough so the redesign team can learn what problems students are likely to face and how to resolve them prior to scaling up to full implementation in all sections of the course. The pilot period is an opportunity for a redesign team to uncover technology issues or any problems that might emerge involving the newly designed assignments or activities. For some institutions, the pilot term also provides a time to collect consistent data on student learning from both traditional and redesign sections that can be compared when consistent historical data are not available. For many institutions, the pilot has provided a time to make sure (1) that important audiences both on and off campus have been informed of changes in the course and (2) that all potential bumps in the road have been smoothed. Overall, a pilot provides the redesign team with a dress rehearsal of the redesigned course and an opportunity to resolve any issues that may arise. Teams have learned that it is much easier to solve problems with 150 to 200 students rather than with 1,000 students. Conduct Workshop III: Mid-Course Sharing After the campus pilots have been completed, program leaders should conduct a one-day, face-to-face workshop that provides a forum for teams to share their experiences and learn from one another. Teams share their initial findings regarding learning and retention outcomes, cost containment and implementation issues. Such interim reporting provides an opportunity for self-evaluation so that teams look carefully at what has gone well, at what surprises have required adjustments, and at what kinds of issues still remain to be resolved. This workshop enables teams to benefit from what others have learned and accomplished and to receive feedback from the group as well as from program leaders. Having a workshop that has been planned from the program’s launch also encourages project teams to keep on schedule because they know they will have to report publicly on their progress to their peers and the program leaders. Teams should be asked to complete two tasks in preparation for the workshop.
The program leaders should review the teams’ work, assess pilot outcomes, and offer suggestions for improvement and adjustments in preparation for full implementation. The program leaders should also meet with teams individually, if needed, to resolve any particular issues they face and have not been able to overcome. A sample invitation to the workshop, which outlines those tasks, is included in the appendices. A sample agenda and a list of the logistical elements of the workshop that program leaders need to accomplish in preparation for the workshop are included in the appendices. Monitor Redesign Plan Revisions Conducting the pilot in the spring term gives the team time during the summer to address issues that may have arisen during the pilot. Inevitably, the redesign plan will need to be tweaked so that any problems encountered can be resolved. The team may have to modify and/or add policies and procedures so as to address issues that emerged during the pilot. Training plans may need additional refinement to include any new policies or procedures that got adopted during the pilot. The team should also check with offices on campus to resolve any difficulties that may have been encountered. Some institutions have conducted focus groups with students to uncover problems that can be corrected during this period. Monitor the Full Implementation One of the goals of course redesign is to include under the redesign model all of the institution’s students who are enrolled in the course. NCAT calls the first term when this occurs full implementation of the redesign. During the fall term, teams fully implement the redesign with all students enrolled in the course and include all aspects of the redesign. All students benefit from the new learning environment, and both students and the institution benefit from reduced costs. Course policies and procedures are consistently applied to all students, and all students have an increased opportunity to succeed. Some modifications of the policies and procedures may have to be made, but they will likely be minimal if the team carefully thought its plan through and made corrections after the pilot. Collect and Review Final Reports After the first term of full implementation, teams should be required to submit final reports to the program leaders by following a consistent format that facilitates comparison among projects. Examples of final project reports are available on the NCAT website at http://www.theNCAT.org/PCR/Proj_Success_all.html. Follow the links to each project listed under Course Redesign Exemplars. The program leaders should collect, review, and verify assessment data and cost data from the institutions. Program leaders need to ensure the validity of the assessment results, the accuracy of costing figures, and overall fidelity of the process. They should meet with teams individually if needed to resolve any particular issues the teams face and have not been able to overcome. A Final Report Format description is included in the appendices. Conduct Workshop IV: Assessing the Results After the first term of full implementation, the program leaders should conduct a one-day, face-to-face workshop that serves as a forum for teams to communicate their experiences and learn from one another. Teams will share their findings regarding learning and retention outcomes, cost containment and implementation issues. Such reporting provides an opportunity for self-evaluation so that teams look carefully at what has gone well and what kinds of issues still remain to be resolved. This workshop enables teams to benefit from what others have learned and accomplished and to receive feedback from the group as well as from program leaders. Having a scheduled workshop also encourages project teams to keep on schedule because they know that they will have to report publicly on their progress to their peers and institutional leaders. Teams should be asked to complete two tasks in preparation for the workshop.
The program leaders may want to open this workshop to the broader campus community so that all campus constituencies can learn about the redesign process and outcomes. A sample invitation to the workshop, a sample agenda, and a list of the logistical elements of the workshop that program leaders need to accomplish in preparation for the workshop are included in the appendices. Conduct a Program Evaluation After Workshop IV, the program leaders should conduct an evaluation of the program and prepare a final report. The report should include an assessment of each funded project as well as a review of overall program outcomes and recommendations for expanding the redesign process in the future. The program leaders may want to prepare one version of the report—which should be candid about both the strengths and the weaknesses of the program—for campus executives and one for the broader campus community, which should highlight the positive results achieved as a basis for building support for future redesign efforts. Both the latter version of the report and individual project reports should be added to the program website. A program evaluation template is included in the appendices. Q: Will we see positive results from the initial redesign implementations? A: You can expect mixed results in improving learning and in completion rates, especially in the pilot and sometimes during the first term of full implementation, although many projects show immediate improvements in both areas. Course redesign involves a major change in academic practice with a lot of moving parts. Consequently, projects often encounter issues that need to be addressed during the initial implementations. NCAT’s experience has been that despite mixed results in initial implementations, the vast majority of course redesign project leaders are fully supportive of the continuation of their redesigns as captured in the sustainability section of individual project reports. The professional judgment of the faculty is that the redesigns are effective in improving the quality of the course. Greater consistency of content and coverage, valid and reliable measurements of student learning, greater student engagement in course content—all serve to back up those judgments. Project leaders are generally confident that learning-outcomes data will improve as they address the issues that arose in the initial implementations. Despite mixed results in improving learning and in completion rates, all projects are usually able to reduce their costs. One of the powerful messages of course redesign is that achieving the goal of reduced cost can have a significant impact on an institution’s ability to deal with budget crises, serve more students with the same amount of resources, and free faculty to do other institutional tasks—all with no diminution in quality. Q: What should program leaders be looking for when monitoring the redesign implementations? What are the most likely problems to occur? A: Unfortunately, things don’t always go according to plan. Three NCAT guides focus on the specifics of course redesign: How to Redesign a College Course Using NCAT’s Methodology, How to Redesign a College-Level or Developmental Math Course Using the Emporium Model, and How to Redesign a Developmental Math Program Using the Emporium Model. The guides try to anticipate most of the issues that arise during a course redesign, and that’s one of the reasons this guide should be used together with those three. Some students resist the new way of doing things, and some kind of logistical, technological, or facilities problem almost always occurs. Again, this is why you need a leadership team that pays attention to the redesign projects during both the development and the implementation periods: so that the right person can help resolve problems in a timely fashion. Like students who won’t ask questions in class, faculty, too, frequently won’t ask for help. That’s why you need to be proactive in monitoring the projects. Q: How should we deal with projects that deviate from their approved plans during the implementation period? A: The program leaders must actively monitor project implementations. Sometimes redesign projects do not follow their plans to improve learning while reducing costs (e.g., they do not intend to fully implement their redesign plans.) When that’s happened, NCAT has dropped those projects from the program and/or requested that unspent funds be returned.
|
 |
Introduction Appendices: A. Plan of Work
|
 |
||||||||
 |
|
|
|
|
|
|
|
|
|
|
|
|