HOW TO ORGANIZE A CAMPUS-WIDE COURSE REDESIGN PROGRAM USING NCAT'S METHODOLOGY

VI. Assessing Course Readiness

The purpose of this stage of the program is to ensure that course redesign teams are created that are clear about what they are trying to accomplish and how they intend to achieve it. This stage, which will take about three months to complete, comprises three parts:

• Require applicants to establish course redesign teams
• Require applicants to complete the readiness instrument as a team
• Review responses to the readiness instrument

Require Applicants to Establish Course Redesign Teams

Program leaders should require those interested in participating in the redesign initiative to establish redesign teams and to think carefully about which courses are good candidates for redesign. Successful course redesign is the product of a team effort. It is neither a faculty project nor an administrative project nor a professional staff project. It takes all of those people—because it is a team effort.

Teams that are well organized understand the amount of work needed for the redesign and allocate the redesign work among members. The teams move expeditiously through the planning and development process—spending their time effectively—and achieve both success and sustainability. In evaluating prior redesign programs, we have found that taking a team approach always receives the highest possible rating from participants. We have also found that projects that did not form project teams tended to struggle and be less successful.
Those interested in participating in the program should establish redesign teams that consist of the following.

• Faculty Experts. Course redesign requires that faculty experts explicitly identify a course’s desired learning outcomes and agree on course content. Most courses appropriate for course redesign are typically taught by more than one faculty member. To ensure course consistency, faculty experts must work together on the redesign—resolving any differences in how the course will be offered—and must collaboratively plan the most effective way to accomplish the redesign goals.

• Administrators. Because redesigns affect multiple sections, large numbers of students, and academic policies and practices, it is important that the team involve academic administrators. The level of those administrators will depend on the organization of the institution and the institution’s size. For some, it will be the provost/academic vice president or designee; for others, it will be a dean or department chair. Those team members play important roles when institutional issues arise such as changes in scheduling or the use of classroom space. If unexpected issues arise in the process of redesign implementation, administrators can help the team resolve them quickly and effectively across institutional offices.

• Technology Professionals. These team members provide expertise so that the redesign goals are accomplished in ways that make the technology as easy as possible for students to use. Technology professionals contribute ideas about how to increase interaction with content as well as with other students. They also suggest design approaches that ensure that the technology will not limit students’ learning options.

• Assessment Experts. In Chapter VII of How to Redesign a College Course Using NCAT’s Methodology, NCAT sets forth straightforward methods whereby student learning in the redesigned course can be compared with student learning in the traditional course. It is, however, useful to include on the team a member who is knowledgeable about assessment and research design—especially if the institution seeks to measure additional facets of the redesign such as performance in downstream courses or student satisfaction. Such expertise may be found in a department of psychology or a department of education or in offices of institutional research.

• Instructional Designers. If your campus is fortunate enough to have instructional designers on staff, you may wish to add one to the team. An instructional designer can help guide the re-sequencing of instruction and provide insight into learning theory and modularization. Subject matter experts are not always learning experts, and such guidance can be crucial.

Why: Part of the goal of the redesign process is to teach institutions how to improve learning while reducing cost, which means involving a variety of personnel at all stages of the projects. Teams are key to successful redesign projects, and all players—not just faculty—should be included in early planning because of the multiple dimensions involved in large-scale course redesign.

Require Applicants to Complete the Course Readiness Instrument as a Team

Some courses may be more ready than others to be the focus of a large-scale redesign effort. Because of prior experiences with technology-mediated teaching and learning, and because of numerous attitudinal factors, some faculty members may be more ready to engage in large-scale redesign efforts to achieve the program’s goals.

Those interested in participating in the redesign program should be asked to think carefully about which courses are good candidates for redesign and to respond to the Course Readiness Instrument as a team, which will be the team’s first activity. Completing the readiness instrument enables each team to assess collectively its strengths and weaknesses, thereby gaining an understanding of what it needs to do to close gaps in its preparation early in the process. No team perfectly meets all of the readiness criteria, especially at the beginning of the planning process. Every team will discover things it needs to work on in order to carry out a successful course redesign.

Why: This exercise is designed to establish the importance of a team effort, to help teams select the appropriate course, to analyze their institutional circumstances so they can be sure they are prepared to launch a successful project and to identify “gaps” in readiness that need to be addressed. Reviewing readiness responses enables the program leaders to assess (1) teams’ seriousness of purpose in the way they complete their responses and (2) their readiness to take on a large-scale project as well as to eliminate those teams that have chosen inappropriate courses.

Course Readiness Instrument

The readiness criteria are designed to help program leaders select the courses to be redesigned with the highest chances of success. In some cases, applicants will be asked to read an article, discuss the reading as a team and make a tentative decision, which may change as they learn more about the redesign process. Answering each as honestly as possible—and providing data to support their answers—will lead to the most positive outcome for each potential project.

Teams wishing to participate in the program should send a narrative addressing each of the course readiness criteria (about one page each) as they apply to the selected course, focusing on evidence that demonstrates the way in which they meet each criterion.

Responses to the Course Readiness Instrument should have a deadline date within the timeline and should be submitted electronically.

Teams should be asked to include a cover page with their proposals on which they

• List all team members by name including titles, academic affiliation, phone numbers and email addresses;
• Identify the person who is the primary contact for the team project, with the understanding that the primary contact will share communications appropriately with the rest of your team.

Review Responses to the Readiness Instrument

Program leaders need to review the responses in order to select teams to be invited to Workshop II: Developing the Redesign Proposal described in Chapter VI. NCAT recommends that reviewers use a three-point scale where each response to the Readiness Instrument is read independently by program leaders and rated 1 (strong response; no outstanding issues or only minor ones), 2 (potentially acceptable pending resolution of outstanding issues or confusion), or 3 (weak response or does not meet program guidelines).

Why: Responses to the readiness criteria provide indicators of how well teams understand the program, how they are thinking about possible redesigns, and how much initial preparation the teams have undertaken. The responses generally will not lead to rejection of a team’s ideas this early in the process. Some of the responses, however, will serve as an early alert or warning that some teams or team members have not totally embraced the goals of the program or that some may need additional explanation of exactly how the process will work. The responses enable the program leaders to send feedback to those submitting responses as well as to clarify or emphasize the program’s goals for those who are not clear at this point or who seem to demonstrate some ambivalence about the goals. The responses also help program leaders advise weaker redesign teams about what those teams need to do to become ready.

In most situations, completing the readiness instrument represents a first pass at ensuring that participants understand the program’s goals and expectations. In cases in which a large number of responses to the instrument are received—as was the case in most NCAT national programs—the readiness criteria may enable the program leaders to make screening decisions. The decision to eliminate a particular proposal that received a 3 rating would be based on things like the team’s selecting the wrong course or making statements that clearly indicate the team has no interest in reducing cost. The program leaders can address those issues individually with teams or simply not invite a particular team to Workshop II.

Readiness Review Criteria

Indicators of strong responses to each criterion are listed next.

1. Course Choice

Evidence that

• The enrollment in the course is relatively high and the course is taught in more than one section by more than one instructor.
• The course faces a clear problem that redesign can solve such as high drop/withdrawal/failure rates, inconsistency among sections, and difficulty finding qualified adjuncts.
• The team intends to redesign the entire course: all sections and for all students.
• Faculty plan to coordinate their pedagogical approach rather than leaving it up to individual instructors to make decisions so that all students have a uniform, high-quality learning experience.

2. Redesign Model

Evidence that

• The team has thoughtfully considered the six models and has made its tentative selection of a particular model based on its readings, its examination of NCAT case studies in its disciplines, and the characteristics of its students.
• The institution and the team can support the choice of model (e.g., sufficient lab capacity for an Emporium Model, sufficient skills and support for the relatively complex Buffet Model.)

3. Assessment Plan

 Evidence that

• The team has established learning outcomes for the course to be redesigned that have been agreed to by all who teach the course.
• The team has already collected baseline assessment data for the course (e.g., the instructors may have used a common final exam in all sections of the course for the past five years and have scores for students) or has a clear description of how it expects to collect the needed data for the traditional course during the pilot period.
• The team has thought about how to collect comparable data in the redesigned course.

4. Cost Savings Plan

Evidence that

• The team has read Chapter V of How to Redesign a College Course Using NCAT’s Methodology and understands and supports the program’s goal of cost reduction.
• The team has decided on a tentative cost-saving strategy. The more clearly and simply the strategy can be stated, the stronger the response.

5. Learning Materials

Evidence that

• The team is clear about the program’s focus—which is not on software development—and is committed to using existing, commercial products as a centerpiece of the redesign.
• The team has investigated or is willing to investigate the range of existing learning software in the selected academic area and to collaboratively select a product that will foster the goals of the redesign. For example, the team may have already made appointments with publishers or software companies or contacted other institutions to learn what they are using in comparable courses that have been redesigned.
• The team sees that software is an integral focus of the course rather than a supplement or add-on.
• All faculty teaching the course will use the same learning materials, collaboratively selected by the team.
• The team recognizes the need for training and mentoring of all faculty—both full-time and adjuncts—teaching the course in how to use the materials.
• The information technology staff is willing to assist the team in installing and maintaining the software and in training and supporting faculty.

6. Active Learning

Evidence that

• The team understands the need to move beyond lecturing to engage students in active-learning activities. Some lecture may be included, but it represents a minor portion of the course; that is, students will spend the majority of class time working in small groups solving problems or actively engaged in using interactive software.
• The team can supply concrete examples of what it will do to foster active learning by way of such statements as, “Low-stakes quizzes will be required twice each week” or “Students will be expected to work in small groups online and then present the results of their work in class.”
• The selected software will include features that enable students to do such things as practice key content ideas and apply principles.

7. Collective Commitment

Evidence that

• A team has been created that includes faculty as well as administrators, information technology representatives, and, if available, instructional designers.
• The team recognizes the importance of collective commitment and its pivotal role in sustaining the redesign.
• The team has achieved consensus among those teaching the course and in the department about the redesign initiative and its goals, the anticipated changes in the course, and the impact on the department.
• The team has alerted other institutional offices—such as advising, financial aid, and the registrar—that the redesign will require their involvement.

 
Q: NCAT requires potential program participants to form teams. Do you ever run into problems in this regard?

A: Ideally, teams should be formed immediately after the first workshop—meaning, very early in the process. Responses to the readiness criteria and workshop homework should be completed as a team. In our experience, sometimes individuals rather than teams have completed those assignments. The worst instance of this approach occurs when a team asks the business office to complete the Cost Planning Tool, a sure indicator that the team does not understand the program’s goals. Without the early formation of teams and consistent participation (e.g., lack of correlation between attendees at the first two workshops), a shared learning experience cannot result. The consequences are weak proposals and weak projects.

Q: Who should review the responses to the readiness criteria?

A: The project leaders (the full team) should read the responses to the readiness criteria and rank them individually. The leaders should then meet as a team to discuss the rankings and the responses.

Q: What should the program leaders do if a team chooses the “wrong” course?

A: This can sometimes happen when a particular team becomes enthusiastic about course redesign, but course redesign is not applicable to the team’s course. Typically, this happens because the course is already so inexpensive that there is no easy way to reduce costs. What motivates the team is pedagogical improvement. The program leaders need to remember that a course redesign initiative is not simply a quality improvement program.

Q: Is it possible to choose the wrong redesign model?

A: Generally, any of NCAT’s six models will work with any academic area—with two important exceptions: First, redesigns in mathematics should be required to use the Emporium Model. (See How to Redesign a College-Level or Developmental Math Course by Using the Emporium Model for a full discussion of the reasons.) And second, we do not believe that the Fully Online Model is appropriate for traditional-age freshman students—with the exception of information technology courses—or for particularly disadvantaged students because such students require a great deal of structure in order to succeed. (See The Essential Elements of Course Redesign for an elaborated discussion of this point.) The students in almost all of NCAT’s fully online redesigned courses (see http://www.theNCAT.org/PCR/model_online_all.htm) have been mature adults for whom the fully online environment has proved very effective.

Q: What should the program leaders do if there is dissension in a department and some faculty members don’t think the redesign is a good idea?

A: You need to assess the seriousness of the objections. It may surprise you to know that some faculty are resistant to change. If this is the reason, you need to persuade the resisting faculty to let the course redesign faculty conduct an experiment and judge its merits based on the data. A redesign that improves learning, increases course completion and reduces costs—supported by valid and reliable data—is hard to argue against. But if there is a bitter division within the department or some kind of ideological issue driving the resistance, it would be prudent to not allow the department to move forward in the application process. Eventually, the department’s composition will change, and the newly fashioned department might participate in future rounds of the initiative.

Table of Contents

Introduction
I. The Critical Components of a Successful Course Redesign Program
II. Assessing Institutional Readiness to Redesign
III. Making Key Decisions before the Program's Launch
IV. Developing a Plan of Work
V. Building Awareness and Capacity
VI. Assessing Course Readiness
VII. Preparing Teams to Submit Strong Proposals
VIII. Selecting Proposals That Will Succeed
IX. Monitoring the Redesign Implementations
X. Maintaining Consensus and Ensuring Sustainability
XI. Building Capacity and Scaling Initial Success

Appendices:

A. Plan of Work
B. Publicity Plan
C. Call to Participate
D. Application Guidelines
E. Workshop I Agenda
F. Workshop II Homework
G. Workshop Logistics
H. Workshop II Agenda
I. Workshop III Invitation
J. Workshop III Agenda
K. Workshop IV Invitation
L. Workshop IV Agenda
M. Final Report Format
N. Program Evaluation