 |
 |
|||||||||||
 |
||||||||||||
 |
NCAT Monograph Improving Learning & Reducing Costs: By Carol A. Twigg Preface On July 15-16, 1999, a group of 20 higher education leaders gathered in Roanoke, Virginia to participate in an invitational symposium on the topic of "Redesigning More Productive Learning Environments." This was the first of the recently created Pew Symposia in Learning and Technology, whose purpose is to conduct an ongoing national conversation about issues related to the intersection of learning and technology. The discussion in Roanoke consisted of two kinds of participants. The first group had practitioners, faculty members, and campus administrators who have undertaken large-scale redesign projects. The second group had national and campus leaders who are intensely concerned about the issues of quality and cost in higher education and are looking to information technology as an innovative resource for constructive change. By blending the macro and micro views of the issues, we hoped to arrive at a common understanding. In higher education, we traditionally have assumed that high quality means low student-faculty ratios and that large lecture/presentation techniques are the only low-cost alternatives available to us. New models are emerging, however, that show that it is possible to improve learning while simultaneously reducing the cost of instruction. We can indeed have our cake and eat it too. Our goal in Roanoke was to examine the validity of the conceptual framework that undergirds the projects being implemented in the Pew Grant Program in Course Redesign. The purpose of this program is to encourage colleges and universities to redesign their instructional approaches using technology to achieve cost savings as well as quality enhancements. Redesign projects focus on large-enrollment, introductory courses. There are, of course, many other ways to improve quality and reduce costs in higher education-some of which use technology and others which do not-including efforts at the program level, the institutional level and the interinstitutional level. Our discussion, like the grant program, had as its focus the course level, specifically the introductory course level for reasons that are articulated in this paper. Among the questions considered at the symposium were the following: What prior experience and investment makes an institution ready to engage in redesign? How are the new learning paradigms organized? How can they be improved? What benefits do they offer for students, for faculty and for institutions? What is the best approach to helping faculty adapt to a different style of educational delivery? What are the elements necessary for successful implementation? Are there best practices that we can draw upon to serve as models for other disciplines and institutions? Will these redesigns scale beyond a particular class or institution? This paper, like the discussion in Roanoke, builds on the existing theory of how to redesign more productive learning environments, which is well established. Much has been written on this subject by Robert C. Heterick, Jr., D. Bruce Johnstone, Charles Karelis, William Massy, Jack Wilson, Robert Zemsky, and me, all of us advancing essentially the same argument. Despite the existence of a good theoretical base, those ideas have had very little impact within the higher education community. A major contributor to this minimal impact is that those in higher education who sincerely want to address this complex issue may not be convinced that it can be done or may not know how to do it. Two things are needed: 1) a comprehensive planning methodology (a roadmap or a cookbook, as some have called it) that can guide course redesign for multiple institutions, and 2) examples of practice that prove the theory. Rather than rehash either the reason for taking action or the theory that points the way, our discussion focused on specific ways to implement the theory, drawing upon the experience of four large-scale redesign projects at Virginia Tech, the University of Wisconsin-Madison, Rensselaer Polytechnic Institute, and the University of Illinois at Urbana-Champaign. By iterating between the planning methodology and actual implementations, our intention was to draw lessons that can be used as a guide to practice throughout higher education. This paper represents the result. QUALITY, COST AND INFORMATION TECHNOLOGY A major problem that continues to confront American higher education is that of rising costs. With the average cost of attendance consuming a substantial portion of the median family income, what is at stake for many Americans is nothing less than the continued viability of the American dream. The stakes are high for higher education as well. Caught in a closing vise between new demands for enrollment and declining rates of revenue growth, colleges and universities must figure out a way to do more with less. Recognizing that tuition increases can no longer be used as a safety valve to avoid dealing with the underlying issues of why costs increase so much, campuses have begun the hard work of cost containment. But after sharpening priorities, sometimes making tough choices in light of those priorities, and asking everyone-administrators and faculty alike-to work harder, campuses are still groping for ways to wrestle costs under control. At the same time, colleges and universities are discovering exciting new ways of using technology to enhance the process of teaching and learning and to extend access to higher education to new populations of students. For most institutions, however, new technologies represent a black hole of additional expense as students, parents, and faculty alike demand access to each new generation of equipment and software. Most campuses have bolted on new technologies to a fixed plant, a fixed faculty, and a fixed notion of classroom instruction. Under these circumstances, technology becomes part of the problem rather than part of the solution of cost containment. By and large, colleges and universities have not yet begun to grab hold of technology's promise to reduce the costs of instruction. Containing costs-and making use of new technologies to help contain costs-requires a fundamental shift in thinking. It requires one to challenge the fundamental assumption of the current instructional model: that faculty members meeting with groups of students at regularly scheduled times and places is the only way to achieve effective student learning. Rather than focus on how to provide more effective and efficient teaching, colleges and universities must focus on how to produce more effective and efficient student learning. Faculty are only one of many resources that are important to student learning. Once learning becomes the central focus, the important question is how best to use all available resources-including faculty time and technology-to achieve certain learning objectives. Rather than asking faculty to work harder, we need to enable them to work smarter. Responsible members of the higher education community have an interest in lowering the cost of instruction as long as such an effort does not result in a reduction in quality. Different stakeholders are interested in reducing costs for different reasons. Some are concerned with reducing the cost to society-such as the level of state and federal allocations to higher education; others want to reduce the cost for students and their parents-such as the level of tuition and fees. Both of those views tend to come from stakeholders outside the institution. Within the institution, there are other reasons for wanting to reduce costs. The belt-tightening activities of the past decade have left many institutions with almost no discretionary funds; life on many campuses has an almost austere quality. Institutions are faced with the pressure to invest more in information technology but many are hard-pressed to find additional funds for such investments. Finally, those in higher education most threatened by the growth of private sector competition need to find more cost-effective methods of operation in order to maintain their position in the new marketplace. Assumptions that get in the way Having said that, a series of assumptions about the relationship among quality, cost, and information technology dominates the current discussion, making it difficult to find a solution to the problem. Three of these assumptions are:
Conversely, controlling costs means reducing quality (for example, relying on large lecture courses); increasing the use of adjuncts, teaching assistants (TAs) and other part-time faculty; or, most drastically, laying off faculty. Very few people in higher education believe that it is possible to increase quality and reduce cost at the same time.
Very few believe that investments in IT can generate a return on that investment, not only in terms of increased quality but also in reduced costs.
This belief generally stems from the fear that IT will be used to reduce costs by reducing human contact (or replacing human contact). This fear gets expressed in a variety of ways: the American Federation of Teacher's recent ad campaign about the "Five-Minute University;" the breakdown-in-community argument; and the no-proof argument-"no one has shown that technology can improve learning." Since education is a human or social practice, and it has primarily been practiced in face-to-face settings, physical contact becomes the primary enabler of learning. In contrast, we will show how redesign using technology-based or learner-centered principles can offer a way out of higher education's historical trade-off between cost and quality. Many experts on the subject have pointed out that moving away from our current credit-for-contact mode of instruction is fundamental. Some approaches employ a greater reliance on asynchronous, self-paced learning modes while others take place in a traditional, synchronous classroom setting but with reduced student/faculty contact hours. Both rely on shifting faculty time-on-task to the technology or lessening the labor-intensive quality of instruction. In each case, they are designed to transfer the locus of activity from the faculty to the student: the focus is on student problem solving and interactive learning rather than on presentation of material. A Strategic Focus In order to have maximum impact and to achieve the highest possible return on one's investment, redesign efforts need to have a strategic focus. Like the Pew Grant Program in Course Redesign, the symposium had as its focus large introductory courses with high enrollments. Why concentrate on those courses? Studies have shown that undergraduate enrollments are concentrated in relatively few academic areas. At the community college level, about 50 percent of student enrollment is concentrated in just 25 courses. The course titles include introductory studies in English, mathematics, psychology, sociology, economics, accounting, biology, and chemistry. Those same 25 courses generate about 35 percent of enrollment at the baccalaureate level. By making improvements in a restricted number of courses, one can impact literally every student in the institution. In addition to having an impact on large numbers of students, there are other advantages of such a focus. First, large introductory courses are good prospects for technology-enhanced redesign because they have a more or less standardized curriculum, outcomes that can be easily delineated, and content over which faculty are less possessive. Second, by targeting those courses, what is widely regarded as a prime area of ineffective teaching-the large lecture course-will be improved. Third, those courses serve as foundation studies for future majors. Successful learning experiences in them will influence students to persist in key disciplines like the sciences. Finally, because those courses are feeders to other disciplines, acquiring a deeper foundation and mastery in them will help students make a successful transition to more advanced study. Finally, introductory courses absorb a significant amount of resources. Despite the common wisdom that packed lecture halls and low-paid graduate teaching assistants equal the most cost-effective way to deal with large numbers of students, those who have examined the matter know that lecture-based courses are not cheap. This is especially true when they are combined with discussion sections-employed by most institutions to give students some opportunity for interaction-as well as laboratories. In many institutions, introductory courses are taught in multiple section models by individual faculty members, quite costly given the large number of sections required. Controlling costs in those courses can result in a significant return to the institution. In order to be successful in redesign efforts, one must pay attention to three critical interrelated elements:
What follows is an elaborated discussion of each of these. INSTITUTIONAL READINESS CRITERIA Not all institutions are ready to engage in large-scale redesign using technology. Experience has taught us that certain institutions more than others have progressed farther along the learning curve of what is required in order to create these new learning environments. Because of their prior investments and experiences, some institutions are, in essence, more ready to engage in successful redesign efforts. What follows is a list of preconditions-or readiness criteria-that must be in place before an institution is able to implement such an effort successfully. The institution must want to reduce costs and increase academic productivity. It is questionable how many institutions really want to reduce or control costs. Many, for example, believe that rich inputs are characteristic of high quality and have built their reputations on that view. Others recognize that increasing academic productivity is key to their future prosperity and have made public statements to that effect. For many institutions the prospect of increased enrollment demands without a commensurate growth in resources is the driving factor. For public institutions, declining state support also contributes to the desire to increase productivity. Some institutions' operating budgets have remained flat at levels that existed in the mid-1980s. They face the challenge of offering quality instruction to a steadily growing student body with limited resources and with reduced staff. To do so, many are investigating the use of technology to achieve more efficient and cost-effective instructional delivery. Numerous institutions throughout the country face similar predicaments. Some, unlike those mentioned above, prefer to hope for better financial times rather than deal with higher education's new economic reality. They are like alcoholics in denial. To be successful in using technology to reduce costs, institutions must begin by owning the problem. Just as the only alcoholics who can be helped by Alcoholics Anonymous are those who want to stop drinking, so too must institutions want to reduce costs in order to take the next step. The institution must view technology as a way to achieve strategic academic goals rather than as a general resource for all faculty and for all courses. Almost every college and university in the country provides some kind of support for faculty to integrate technology into teaching and learning. Most, however, stop there without thinking more deeply about how the use of technology enables the institution to achieve its strategic goals. Fewer still focus on specific elements of the curriculum to achieve maximum impact. Does your institutional strategic plan differentiate between general support for faculty and students and strategic applications of technology in the academic program? Several universities have made integration of IT into the teaching and learning process a central strategic goal. Such integration has strong support from both faculty and campus executives. In each instance, the campus has gone beyond crafting an IT plan, to thinking about IT in the context of institutional planning. A few campuses have moved to target specific, strategic parts of the curriculum. While continuing to provide general support for instructional technology, these institutions have taken an important step in moving beyond the support-whoever-walks-in-the-door approach that characterizes most campus efforts. In contrast, campus-planning weaknesses can easily be spotted when generalities predominate planning statements. Many campuses express the desire to integrate appropriate technology into the academic program without defining what is appropriate. Other seek to use technology to achieve academic goals without making explicit what those goals are. Some want to reconceptualize undergraduate education but are woefully silent when it comes to defining how they will do it. Many see technology use as a means to encourage collaboration as if collaboration like innovation is an end in itself. Collaboration for what purpose? To what end? And almost everyone wants to use technology to support excellence. The institution's goal must be to integrate computing into the campus culture. Ubiquitous networked computing is a prerequisite to achieving a return on institutional investment. One really does have to spend money to make money. The University of Illinois at Urbana-Champaign, for example, describes itself as a computing-intensive campus. What characterizes a computing-intensive campus? Anatole France said, "Those who don't count, don't count." In this arena, his comment has merit. Unlike many institutions who have established initiatives without specific milestones, computing-intensive campuses know the numbers. They know the level of network access and personal computer ownership (or availability) for students and faculty on their campuses because their goal is saturation; the numbers tell them how far along they are in achieving that goal. The institution must have a mature information technology (IT) organization(s) to support faculty integration of technology into courses or it must contract with external providers to provide such support. How does one characterize a mature organization? It means that the IT organization can provide more than technical support. It has an understanding of the goals and objectives of the institution's academic program-it can see the big picture. More advanced IT organizations include instructional design capabilities and have specific experience with supporting course redesign. Not all campuses need to develop their own in-house units. Another approach is to contract with one of the growing number of external service providers who have specific expertise in developing online learning environments. Campuses today no longer develop their own administrative applications. Instead, they turn to contractors, not only to develop sophisticated and integrated modular administrative systems, but also to help implement and manage such systems. Before starting down the path of growing their own instructional products and services, campuses should think carefully about whether to build or buy. They should also take care not to confuse technical support with instructional design support, whether its source is on or off campus. A substantial number of the institution's faculty members must have an understanding of and some experience with integrating elements of computer-based instruction into existing courses. Some faculty may have a great deal of enthusiasm for large-scale redesign but little prior experience in this area. It is difficult to complete a successful large-scale redesign project by starting from scratch. Having experience with integrating smaller IT elements into courses helps faculty to prepare for large-scale redesign efforts. Some experts have said that 13 to 15 percent of the faculty constitutes critical mass. Once again, knowing the numbers matters. Less-developed campuses can only cite goals and plans for faculty involvement or participation in training and development workshops without any clear idea about how such training experiences translate into new kinds of learning experiences for students. The institution must have a demonstrated commitment to learner-centered education. What are some indicators of institutional commitment to learner-centered education? Implementing teaching-learning models where (1) the locus of activity has shifted fundamentally from the instructor to the learner and (2) student engagement independent of time and location is not only permitted but also promoted would be one example. Non-technology-based commitments to student-centered learning also constitute evidence. Community colleges often have a clearer commitment to learner-centered education than other sectors because of the emphasis of their missions. For example, part of Miami-Dade Community College's mission is to "provide accessible, affordable, high quality education by keeping the learner's needs at the center of the decision-making process." The College recognizes that students are different and, therefore, should have available a variety of modalities that support their academic, personal and career development. Some institutions demonstrate their commitment to learner-centered education by pointing to the range of pedagogical practices they use to address the variety of student learning styles. These include distance learning, self-paced modules, learning communities, and collaborative learning. Others show their commitment to learner-centered education by providing anytime-and-anywhere connectivity to a virtual learning environment, enabling both traditional and distant learners to access teaching and learning resources, tools and student information. Even though the entire campus may not have embraced a learner-centered viewpoint, different types of indicators can be found that show movement in that direction. Promoting active learning approaches through faculty retreats and internal funding programs are examples of ways to involve dedicated and experienced instructors in the diffusion of learner-centered approaches. The institution must have established ways to assess and provide for learner readiness to engage in IT-based courses. Learner readiness involves more than access to computers and to the network. How computer literate and network savvy are your students? Does the institution have processes in place that enable them to gain these competencies if they are lacking? Readiness also involves access to support for such things as using navigation tools and course management systems. In addition to technical proficiencies, students need to be aware of what is required to be successful in technology-intensive courses. Does the institution have processes in place that assist them in making wise choices and that prepare them for success? Making a major change from face-to-face instruction to online learning involves far more than learning to use a computer. Like all of us, many students are set in their ways after a lifetime (albeit brief) of passive instruction. They need preparation in making the transition to more active learning environments that are technology based. Some students instinctively flourish in those new environments while others require direct intervention and assistance from faculty and staff. Among the possible ways to assure learner-readiness are:
The institution must recognize that large-scale course redesign using information technology involves a partnership among faculty, IT staff, and administrators in both planning and execution. Substantive changes in the way courses are offered cannot rely on faculty initiative alone. They are systemic and involve changes in such institution-wide areas as policy, budgeting, administrative procedures, and infrastructure. Institutional policy regarding things like class meeting times and contact-hour requirements will require revision. In some instances, obtaining governance approvals may be a prerequisite. While innovative faculty members have developed new methods and materials over many years, administrative initiative is required to bring development opportunities to the attention of the wider faculty and to provide the infrastructure and support that enable people to commit their time to course redesign. Curriculum oversight committees must learn to expect and encourage innovative course designs that break the traditional mold by providing flexible scheduling and contact requirements. In many cases, traditional budgeting processes do not welcome innovation and may need to be changed. Registrarial procedures such as registration and classroom assignment systems may need to be adjusted. Redesign may also require additional or unusual equipment purchases and deployment. Personnel policies regarding how instructors of nontraditional courses are compensated may require revision. How an initial large-scale redesign might benefit other courses in the institution also needs to be considered. The lesson of successful redesign is that faculty and administrative collaboration is required even in the planning stages. Institutions that have not recognized this interdependence view redesign as primarily a faculty matter-frequently as an individual faculty member's task for his or her particular course-with some support from the IT organization. Such a view will inevitably resign institutional advancement to, in Bill Graves' apt phrasing, "random acts of progress" rather than substantive accomplishment. And inevitably these efforts will be under-supported and incapable of generating a return on institutional investment. Just as some institutions are more ready than others to engage in large-scale redesign, some courses are more ready than others to be the focus of that redesign effort. Because of prior experiences with technology-mediated teaching and learning, and because of numerous attitudinal factors, some faculty members are more ready to engage in large-scale redesign efforts. They have, in essence, a head start on the process. What follows is a list of preconditions that must be in place in order to identify a particular course as a successful redesign candidate. Improvements in the course potentially must have a high impact on the curriculum. Is the course a large introductory, high-enrollment course? Is it taught regularly? Is there a significant academic problem in the course such as a substantial failure rate? Does the course face a serious resource problem such as how to manage increased enrollment demand with no commensurate increase in resources? The course must offer the possibility of capital-for-labor substitution. Large size per se does not necessarily make a course a good candidate. The University of Illinois at Urbana-Champaign, for example, offers an introductory comparative literature course that enrolls about 250 students a semester. It is writing intensive and satisfies the campus composition requirement. In spite of the course size, the possibility for capital substitution is limited. Competent evaluators must assess the students' written work that is contextually based, thus limiting the possibility of capital-for-labor substitution. Bill Massy has suggested that IT has strong potential to increase learning productivity in areas of codified knowledge and algorithmic skills. Examples of good target subjects include remedial and basic math and other basic general education courses. In those specific areas, the implication is that IT should supplement human instructors whenever possible-human intervention should be oriented mainly towards making the advantages of IT accessible to all learners. Decisions about curriculum in the department, program, or school must be made collectively-in other words, beyond the individual faculty member level. Decisions to engage in large-scale course redesign cannot be left to an individual faculty member. He or she may leave the institution, grow tired of the innovation, change his or her mind, and so on. A collective commitment is a key factor for sustainability of a redesign project. Indicators that the faculty in a particular unit are ready to collaborate include the following: They may have engaged in joint conversations about the need for change; they may have decided to establish common learning objectives and processes for the course in question; they may have instituted pieces of a common approach such as a common final examination. Institutional support is important, but departmental ownership of the course redesign idea is essential. If one wants to institute change, the best chance of success is to have a group of people who are committed to the project's objectives working together rather a single individual. This is even more important when it comes to sustaining change. The faculty must be able and willing to incorporate existing curricular materials into the project in order to focus work on redesign issues rather than on materials creation. Disciplines with a comparatively large existing body of technology-based curricular materials and/or assessment instruments are especially appropriate targets. The studio course model at Rensselaer relied initially on materials created by the nationwide CUPLE project. The chemistry redesign project at the University of Wisconsin-Madison builds on decades of collaborative work in chemistry software development. Faculty who are willing to employ an appropriate blend of home-grown (created by local faculty) and purchased learning materials in a non-dogmatic fashion will also have a head start. For example, Rio Salado is redesigning a mathematics course around Academic Systems interactive software that students purchase for themselves. In its Math Emporium project, Virginia Tech first tries to locate existing materials to incorporate into its courses before turning to materials creation. Faculty who are subject to the not-invented-here syndrome, who believe that they must create everything themselves from scratch, will be consumed with materials development and add large amounts of time to the process. Those who are willing to partner with other content providers, whether commercial software producers or other universities who have developed technology-based materials, make better candidates for a large-scale redesign project. Project participants must have the requisite skills. Does the potential project have strong leadership? Champions or heroes frequently play a significant role in redesign projects. People need concrete evidence that an idea is doable as a way of starting, and having one person who can do something that others can see is important. While the hero may be needed at first, the innovation will not grow and be sustained if the project continues to rely on heroism. Large-scale redesign efforts almost always will involve partnerships between faculty, IT staff, and others. Each of the parties in a redesign project must have the requisite skills (i.e., they must be competent to do the job) and they must be prepared to partner with others when necessary. Is there evidence, for example, that the faculty and staff involved are ready to move the project forward in a timely manner? Successful projects build on an established skill set. For example, faculty who have some experience with computer-based instruction beyond putting syllabi on the Web are more likely to succeed than others. Such experiences include developing outlines and storyboards for pilot modules; developing computer-based tutorials and diagnostic quizzes and assignments keyed to questions in the quizzes; and using course management systems that facilitate student-to-student and student-to-instructor communications. The course's expected learning outcomes and a system for measuring their achievement must be identified. Successful large-scale redesign efforts begin by identifying the intended learning outcomes and developing alternative methods other than lecture/presentation for achieving them. The curriculum is then built backward from the intended outcomes. Many redesign efforts take advantage of national standards and normed assessment instruments in their particular disciplines as a framework for structuring the project. Many campuses have established an assessment culture, making it easier for them to assess the learning outcomes of innovative projects as well as for those of traditional courses and programs. Does the campus have assessment processes in place, such as the ability to collect data, the availability of baseline data or the establishment of long-term measures? Is there a system for measuring the achievement of outcomes at both the individual student level and the class level? The faculty members involved must have a good understanding of learning theory or access to expert partners. Sound pedagogy is the key to successful redesign projects. When sound pedagogy leads, technology becomes an enabler for good practice rather than the driver. Does the instructor seek to use technology to transform the teaching and learning environment to achieve learning improvements rather than merely to automate existing instructional practice? Has the instructor systematically thought about and investigated alternative methods for empowering students to learn? Faculty who already provide a range of options for achieving required learning outcomes are especially good candidates. Frequently, one assumes that university faculty have an understanding of learning theory simply because they are teachers. In reality, many are exposed to these ideas for the first time during faculty development experiences. Through working in partnership with instructional designers, faculty can become knowledgeable about learning theory and its relationship to course design. In order for the innovation to be self-sustaining in the future, one must have a business plan to support the ongoing operation of the redesigned course. In order to be sustained, changes in instructional practice must be affordable by institutions and integrated into their base funding practices. A wealth of experience shows that attempts to add on innovations with external support and without internal structural change-especially commitment of resources in the institution's core budget-have been almost totally unsuccessful. When the grant funding runs out, the innovation ends. The most surefire way to tell whether an innovation is for real or is artificial is to look at its funding. Unless an innovation is paid for directly by those who stand to benefit from it, its chances to flourish are dubious at best. After determining that the institution is ready and selecting an appropriate course, the next planning step is to identify the academic problems that the redesign intends to address. Most of the weaknesses attributed to large introductory courses are generic in nature and have as their source the limitations of the predominant form of instruction in our nation's colleges and universities, the didactic lecture. The overwhelming body of research tells us that students do not learn effectively from lectures (Halloun & Hestenes 1985; Thornton 1990; McNeal & D'Avanzo 1997; Mazur 1997; Seymour & Hewitt 1997; NISE 1999.) Testimony from the field corroborates the literature. What's wrong with the lecture? The lecture method is a push technology. It treats all students as if they were the same-as if they bring to the course the same academic preparation, the same learning styles, the same motivation to learn, the same interest in the subject, and the same ability to learn. The reality is that students with weak skills need more individual attention and more opportunity for interaction, particularly at the beginning of the semester. At the same time, students with strong skills who would benefit from having more opportunity to explore the material fully or who could accelerate are locked into a fixed time frame for completing the course. The large, impersonal lecture format simply cannot accommodate the broad range of student differences. Because the lecture method is largely a one-way technology, it is impossible to employ a variety of sound pedagogical techniques. Most lecture courses are notoriously ineffective in engaging students. The traditional format neither encourages active participation nor offers students an opportunity to learn collaboratively from one another. Relying heavily on reading assignments leads to inadequate student interaction with learning materials, a particular deficiency in those subjects requiring hands-on experience. This one-way methodology does not provide adequate tutoring assistance; consequently, students receive little individual attention. Even though individual help may be available in office hours, only a small fraction of students take advantage of them. Most students simply study the text, turn in their homework, and take quizzes and exams. Adding to the lack of feedback is the way in which most test answers are graded: students receive only the total score and never know what material was incorrect or where to learn the correct information. As a result, in many institutions attendance at large lecture sections averages approximately 50 percent compared to attendance in moderate-sized sections, where it may be better than 75 percent. Some students drop out of the course while others stay registered but essentially give up and stop working after the deadline to drop the course. While success rates vary by institutional type and by the subject matter of the course, Research I universities commonly cite a 15 percent rate of drops, D grades and failures. Comprehensives report success rates (a grade of C or better) ranging from 78 percent to 55 percent in these courses. Community colleges frequently experience retention rates of 60 percent or less. Clearly there is a great deal of room for increasing student achievement levels in these courses. Even more important, those who pass often do not retain much of the material for future use in other courses. All institutions report the inability of students to retain what they have learned in large lecture courses and, more specifically, the inability of students to apply principles learned to other disciplines. Lee Shulman has described those learning problems as the "epidemiology of mislearning" or the "taxonomy of pedago-pathology." Students forget what they learned (amnesia); they don't understand that they misunderstand what they learned (fantasia); and they are unable to use what they learned (inertia). There are other nongeneric problems connected to the lecture method including lack of accessibility, special problems in serving English as a second language learners and the inconsistent quality of TA support. Many student populations are differentiated by on-campus residency and commuting status. The lecture format presents problems to students who cannot attend on a regular basis. Institutions with a large proportion of English-as-second-language students face special problems since those students often have an especially difficult time with courses requiring considerable language facility and use of technical terms, particularly when they are taught in a stand-up format. Conversely, many teaching assistants who provide the direct contact with students in recitation sections and labs are not native speakers of English and often find it difficult to communicate with beginning-level students. Other large lecture courses are forced to rely heavily on TAs who do not have undergraduate degrees in the field of study, thus limiting the effectiveness of the instruction they can provide. Finally, many courses have no TAs at all and consequently offer no resources to support learning other than the large lecture. Finally, the primary alternate structure for large-enrollment courses, the multiple section model, suffers from problems of its own. While allowing greater interaction with students in theory (although in practice these sections are often quite large and are dominated by the same presentation techniques), the multiple section model suffers from a lack of coordination. Whether taught by tenured faculty, instructors, adjuncts, or graduate teaching assistants, this model requires each instructor to develop his or her own set of course materials, including tests and examinations, and to deliver what is basically the same material in his or her own style. As a result, course outcomes vary considerably and, more important, are not always consistent with students' abilities. Virginia Tech has found, for example, that multiple sections with no common approaches produce a remarkable lack of uniformity in outcomes. Course grades across sections often bear surprisingly little statistical relation either to input-such as SAT profiles-or to scores on the common final exam. Finally, when courses are taught this way, there is no ability to pool the intellectual resources of the faculty to achieve the best course design and to institute continuous improvement practices. The irony of this situation is that lectures are used most frequently in introductory subjects with the weakest students (read nonmajors) while small, interactive courses predominate in those courses with the strongest students, (for example, the senior seminar). The goals of redesign The purpose of course redesign is to improve student learning. Consequently, redesign must adhere to sound pedagogical principles. Just as there is a substantial body of knowledge about the limitations of the didactic lecture, so too is there ample evidence about those pedagogical principles that result in increased student learning. Much is also known about the role of information technology in supporting those principles of good pedagogical design-what works, and what does not work. If instructors merely add on technology to ineffective instructional methods-if they simply technologize the lecture method-there will be no improvement in student learning. Conversely, while it is true that teachers using paper-based systems can employ effective instructional methods, such an approach tends to increase the teaching's labor-intensive quality. To illustrate this point, Jack Wilson has described how a faculty member at Rensselaer tried to demonstrate that he could teach an introductory calculus course using all of the positive pedagogical techniques utilized in the studio method but without using technology. After two years, he was burned out and begging for a sabbatical. In addition, there was a high institutional cost for the service of the four TAs needed to grade his worksheets. Furthermore, his model was not diffused; no one adopted it. Good pedagogical practice enhanced by technology supports shifts in the nature of the teaching-learning enterprise, making it more active and learner-centered. Alternatives that improve quality involve, among other things, shifting repetitive tasks from instructors to IT-based resources and developing IT-based interactive materials. Technology can be deployed to optimize sound pedagogy by making it more consistent, by providing additional practice and examples and rapid performance feedback, and by making more instruction available on-demand. Technology can provide tools that replace, augment, or extend the ability to identify, collect, organize, integrate, and generate knowledge. Technology can also support models and approaches that change in kind the nature of the teaching-learning enterprise. In effect, the new approaches and mechanisms stand as a new paradigm for student learning. Some subjects are particularly well-suited to computer-mediated techniques. A large part of the content of many introductory courses consists of codified knowledge that must be mastered before more complex systems can be understood. Introductory level mathematics, for example, typically involves a modest conceptual core, underpinning a great deal of numerical and symbolic calculation. Interactive computer instruction is a natural way to provide examples and practice in implementing the ideas, especially where practice efforts and repetition count toward mastery of content. Those subjects that require hands-on experience with data analysis and collection such as statistics and other research-based disciplines can easily take advantage of available technologies as a way of teaching concepts and techniques. Most statistical skills can be practiced and evaluated on the computer. Any portion of a course that concentrates on skill acquisition can benefit from an IT format. Other subjects that are particularly well suited for technology-mediated learning include those that are visual in nature-where many of the concepts are illustrated by images. Because many students at the introductory level are nonmajors, they have an especially great need to see the material that is taught. Biology, for example, is rich in graphics and uses many visual cues. Its many phenomena are good subjects for animation that can capture essential or otherwise unobservable parts of the phenomena. Astronomy is another such example. Much of current astronomical research is carried out through analysis of images. The Web is already rich with astronomical images, animations, and Java applets to illustrate its concepts. With technology resources and a good roadmap, students can be given a rich exploratory experience using materials that are already available. Most redesign projects involve significant structural change. Some eliminate some lectures; others eliminate all lectures. The premise is that faculty do not need to spend as much time (or any time) presenting information. Lectures are replaced with a variety of learning resources, all of which involve more active forms of student learning or more individualized assistance. Redesign involves moving from a push strategy, which presents all material to all students in the same way and at the same time regardless of their particular needs, to a pull strategy. Students access the material they need when they need it, an approach that takes into account differences in learning styles and abilities. The latter strategy is not only more effective in dealing with learning issues; it is also more economical in dealing with resource issues because students use only as much resource as they need. The primary goal is to shift students from a passive, note-taking role to an active learning orientation in order to enhance learning outcomes. As one math professor puts it, "Students learn math by doing math, not by listening to someone talk about doing math." In moving from an entirely lecture-based to a student-based approach, learning is less dependent on the instructor conveying words and more driven by students reading, exploring and solving problems. Students are responsible for their learning in that they are expected to construct their own knowledge by working individually, with other students, and with instructors. What are some of the more active forms of learning that replace presentation formats? First, Web-based resources that have as their goal greater student engagement with the material are one example. Interactive tutorials and exercises give students needed practice; computerized, low-stakes quizzes provide immediate feedback, repetition, and reinforcement; technology-based materials teach abstract concepts interactively. Technology-based instruction enables students to explore material independently and can be structured so that students are compelled to keep up with reading and lecture material (self-paced with deadlines). Because the materials are always available online, students have opportunities to refresh their knowledge when needed. Second are techniques that enable greater individualization of instruction. By providing 24 x 7 access to online learning resources, these new environments are both accessible and flexible and allow students to study at times most convenient to them. By using self-paced interactive learning materials that include diagnostic self-assessments, for example, the more self-motivated and more advanced students can rapidly move ahead on their own, while the students with less aptitude and knowledge can adjust their learning progress more appropriately. Learning many introductory subjects typically involves reaching a threshold of understanding, over a period that varies widely among students, after which the ideas seem easy. IT provides an individualized setting that is well suited to this learning pattern. Greater individualization can also be achieved by modularizing materials and incorporating examples from various disciplines so that students with different professional and personal goals have a greater variety of learning resources. If students are having difficulty understanding a particular part of the course, it can be changed and improved in real time. By moving presentation of material and assessing student mastery of it to the technology, faculty and teaching assistants are freed to spend more time providing individualized assistance where and when it is needed. A goal of redesign is to allow faculty to spend more time with student questions and necessary student intervention and less time delivering content. In this setting, faculty are able to have more one-on-one contact with individual students and thus can more readily address the different needs of individuals. Students can be challenged according to their own skill levels. It would be a mistake to assume that technology-based redesigns consist of individual students sitting in front of screens. Many redesigns incorporate collaboration as a central feature. The University of Colorado at Boulder, for example, plans to redesign its introductory astronomy course using technology. A key feature of the redesign will be to divide the students into learning teams of nine each. Each team will meet with an undergraduate coach in a dedicated computer classroom for two hours per week where they will explore computer-based materials, discuss the posted questions, help each other with homework problems and prepare challenge questions for the rest of the class. Students will be graded not only on their individual performance, but also on the performance of their learning teams. Learning teams will earn team credit for their responses to questions, for participation in dialogue, and for posing good challenge questions. Taking as its cue the use of teams in leading MBA programs, the University of Illinois at Urbana-Champaign is redesigning an introductory economics course by having students work in teams throughout the semester. Scheduled discussion sessions will be used for team meetings rather than for presentation of content in order to ameliorate the matching schedule problem. Student teams will receive training not just in how to use the various software in the course, but also on how to make the teams function most effectively. Once the teams are formed, all work other than examinations will be assessed on a team basis. Penn State plans to take a similar approach in its planned redesign of statistics. With multiple options for learning at their disposal, all students are able to concentrate their efforts on those materials and pedagogies that best match their learning styles. By enabling students to take control of their learning through technology-mediated materials, a single course can more effectively serve a broad range of student learners. But do they really learn? When planning to assess large-scale redesign projects, one must distinguish between impact-finding out whether the ultimate goals of an alternative method (better learning at lower cost or the like) were accomplished-and implementation-finding out whether the institution actually did what it said it was going to do. In any innovation/action research project, both are important. To assess impact, it is important to be clear about what one is after conceptually. The real premise of the redesign strategy-and that which ought to be assessed-is not just learning but learning productivity. This conceptual framing of the dependent variable has several important implications. First, the perspective of analysis by definition consciously compares the innovation(s) against an established baseline of current practice. This implies collecting data on learning and costs in both innovative and comparable standard practice settings, such as comparing outcomes directly for a technology-enhanced course with those of a course delivered in the usual way. Second, the variables used in the comparison to define impact need to include both learning outcomes and later behavior-things like retention rates, course completion rates, and the like-as the latter can significantly impact the learning productivity equation even if learning outcomes are equivalent. Peter Ewell, Senior Associate at the National Center for Higher Education Management Systems, suggests four specific assessment techniques to answer the question, did they really learn? Matched Examinations. Usually these are final examinations administered in common to students completing both technology-intensive, redesigned sections and traditional sections of the same course. This classic evaluation design is the one that has been used the most in assessing both technology-based and distance-delivered courses. Because it is often hard to get agreement on common exams among different faculty, it is possible just to use a portion of the exam to contain some common items. It is also useful to have collected some baseline information on the participants to see if the treatment and control groups are really similar, such as prior GPA, major field, and demographics. Student Work Samples. This is a kind of fall-back position from the former that relies on existing student products, such as papers and problem sets, produced as a part of each course. One needs to choose only a few examples, such as one term paper and a short essay or a lab report, or a synopsis of an article that is roughly comparable, although one virtue of this technique is that the items do not have to be completely similar. This too can be done on a sample basis rather than collecting everything. Once a reasonable sample (n=20 or so) from each class is assembled, the pieces can be cross-scored by a reading team using a scoring guide to look at things like communications ability, mastery of particular areas of knowledge, and so on. Behavioral Tracking. This approach relies on following students who were enrolled in parallel sections (innovative and traditional) through student records to see what happened to them later. Several dimensions of behavior are especially useful to look at here, including:
Attitudinal Shifts. This approach relies on student testimony about changes in their own levels of confidence about the aterial and/or their motivation to continue in the subject area. Again, it is not satisfaction in the feel-good sense but rather focuses on attitudes that are likely to be directly related to subsequent persistence and time on task. For example, INSTRUCTIONAL TASK ANALYSES AND FINANCIAL PLANNING Since the major cost item in instruction is personnel, we know that reducing the time faculty and other instructional personnel spend and transferring some tasks to technology-assisted activities is the key to cost savings in instruction. If we can reduce the number of hours spent by faculty and others while keeping credit hours constant with no diminution of learning results, we can reduce costs while maintaining quality. Of course, it is possible to reduce contact hours and save money, but without the use of IT and the redesign of the instructional process, quality would most certainly decline. With technology, one can serve the same number of students at a lower cost-and serve them more effectively. In redesigning large-enrollment courses, there are, of course, a variety of ways to reduce costs and, consequently, a variety of instructional models that can be developed depending upon institutional circumstances. One approach is where student enrollments stay the same but the instructional resources devoted to the course (course expenditures) are reduced. This approach makes sense when the demand for the particular course is relatively stable. Another approach is to increase enrollments with little or no change in expenditures. This technique is appealing to institutions that face greater student demand than can be met using conventional methods. A third way is to reduce the number of course repetitions required to pass a particular course. In many community colleges, for example, it takes an average of 2.5 enrollments to pass introductory mathematics courses. This means that the institution and the student must spend 2.5 times what it would cost to pass the course on the first try. In each of these cases, a translation of the savings to cost-per-student can be used for comparative purposes. In the case studies included in this paper, the University of Wisconsin-Madison intends to maintain the same student enrollment in general chemistry while reducing the instructional resources that are devoted to the course. The University of Illinois at Urbana-Champaign plans to increase enrollments with little or no change in expenditures. This technique is appropriate given the high student demand in Spanish. Without this redesign, the university would not be able to serve its students adequately using conventional methods. Virginia Tech intends to do both: to increase student enrollments while decreasing the resources devoted to the course. Rio Salado College expects to increase enrollments in a distance learning course while simultaneously reducing the number of repetitions required to pass their introductory mathematics course. A translation of the savings to the cost-per-student allows one to compare the results of each approach. Regardless of the particular method employed, a necessary first step is to look at the different instructional tasks that are involved in course design and to analyze the cost of both the traditional method of instruction as well as the new method of instruction utilizing technology. This analysis utilizes activity-based costing, a process generally regarded in higher education as difficult and unpopular. One must go through the process of activity-based costing in order to gain a clear understanding of the source of both costs and potential savings. Doing activity-based costing is not too difficult at the course level, particularly with a prestructured format to facilitate the process. Activity-based costing involves the following steps.
(Please see the UW-Madison spreadsheets and chemistry case study for an illustration of this process.) Instructional tasks fall into two categories: course development and course delivery, both of which are performed by a variety of kinds of instructional personnel. There are four major activities associated with developing a course: curriculum design, materials acquisition, materials development, and preparation/training of instructional personnel. Curriculum design and development involves planning the overall structure of the course including establishing what the students will know or be able to do if they successfully complete the course; identifying the major topics and learning activities of the course and their sequence (course syllabus); and determining how and on what basis student learning will be evaluated. Materials acquisition entails evaluating and acquiring all learning materials, both paper-based and computer-based, that can be purchased or borrowed from others-in other words, those not developed by the faculty member(s) teaching the course. Materials include cases, workbooks, texts, instructional software, course-management software, diagnostic assessments and other kinds of tests and evaluations. Materials development consists of the time spent by the faculty member(s) and all others associated with course development (TAs, professional staff) to develop the learning materials used in the course including lectures and other classroom presentations, assignments, diagnostic assessments, tests, and other forms of evaluations. If the course is supported by TAs or adjuncts, supervising faculty must spend time orienting and training them. TAs frequently must attend lectures as part of their preparation to lead recitation sections. Staff meetings related to the specific course may be required for courses involving more than a single faculty member, such as team-teaching. Instruction and evaluation are the activities associated with the delivery of the course. Instruction encompasses in-class and out-of-class administration of assessment instruments; presentation of materials, (lectures); nonpresentation interactions with students (in-class discussions, office hours, and laboratory sessions); and time spent monitoring the progress of individual students. Evaluation activities include test proctoring and out-of-class time spent grading assignments (homework, labs, exercises), tests, and evaluations. By analyzing the amount of time each person involved in the course spends on each kind of activity, one can identify the cost factors that can be altered. It is obvious, for example, that evaluating and selecting textbooks is less labor intensive than developing learning materials from scratch. Structuring a redesign around interactive software, preferably purchased or borrowed, radically changes the amount of time that must be spent by faculty and/or TAs in class preparation and routine student interactions. Computer-based quizzing eliminates the time spent in grading exercises. The analysis allows you to do detailed planning, including what-if scenarios, to arrive at the optimal mix of time to be spent by the appropriate personnel. Comparing operational costs This planning model compares operational costs: the before costs of the traditional course and the after costs (forecast of what the course will cost when it is fully operational, say for example, in its third offering). In other words, it asks one to plan what the redesigned course will look like at the end of the developmental process. It does not include the up-front develop-mental costs of either the traditional or the redesigned course. There are two reasons for this approach. The first is that the goal of planning is sustainability-how to sustain redesign efforts over the long haul-and sustainability results from permanent changes in operating expenses. The second is that while the developmental period for course conversion has costs associated with it, the amount of those costs can vary widely and can be handled quite differently. They can be paid for from one-time allocations, such as grants from foundations, federal agencies, or the institution, and/or they can be amortized over any number of years. If institutions can see that they will ultimately realize a return on their investment, they will have an incentive to make the needed developmental investment. Then, of course, questions about how much to invest for how much return come into play. Some development costs are not a one-time thing; there will be ongoing development costs as new material comes along. Ongoing developmental costs are part of traditional course formats as well, so some faculty time will be devoted to development in either model and should be included in the appropriate task category. When additional personnel-such as programmers or database managers-are required, those costs need to be accounted for as ongoing operational costs, not developmental costs. Institution-wide support services and administrative overhead are not included in the comparative cost analysis. The assumption is that those costs are constant-they are part of the campus environment-for both the traditional and redesigned courses. Those costs are not being ignored; they apply regardless of the course design so they do not need to be counted twice. Excluding campuswide infrastructure costs Campuswide infrastructure and equipment costs are not included unless the item is specific to the particular course. Campus networking, site licenses for course management systems and desktop PCs for faculty, for example, are part of the campuswide IT environment. (Software, equipment and professional staff that are particular to the specific course, however, are included.) This point deserves further elaboration because it often raises questions in the minds of non-IT executives. There are three reasons for not including the cost of infrastructure in the planning model.
Some believe that savings in instruction cannot compensate for research and administrative investments which, at most R1 universities, is funded in large part out of ICR funds. These campuses look to the research return-in prestige and future ICR-as a primary justification for the investment. Administrative computing is a second huge factor. Gains from instruction are tertiary and quite distant from the first two. If the point of the planning methodology is to rationalize the overall investment by gains in instruction with an audience of campus administrators, it will not be a convincing argument. The counterargument to this analysis is as follows. On most R1 campuses, the research budget is 10 to 15 percent of the total and the administrative budget is 5 to 10 percent (subject, of course, to the criticism that nobody really knows how much goes into anything because of hidden cross subsidies). It is the classic 80/20 problem: 80 percent in instruction and 20 percent in the rest. The rest (research and administration) take a significant interest in the bottom line, but generally instruction does not because such a large percentage of its costs are presumed to be fixed-salaries, building maintenance, and student support services. (In point of fact, 25 years ago similar arguments were made about research and 40 years ago they were made about administration). If those costs are fixed, then the argument might hold together. If instructional costs are not fixed, then a 10 percent reduction in those costs is worth four times as much as a 10 percent reduction in administrative and research costs. Small improvements in instructional costs can yield large returns. We ought to be looking for cost improvements where the costs are highest. What about distance learning? This costing model is equally applicable to distance learning courses. There is a widespread assumption that distance learning delivery is inherently cheaper than classroom delivery. In announcing plans to develop the Western Governors University, for example, Colorado's Governor Roy Romer was quoted as saying, "This is a revolutionary idea. Many people can't afford the traditional way of getting a higher education degree, which is learning by sitting in the classroom. Technology can be an effective and cheaper way to help people learn." Governor Romer is right: technology-based delivery can be cheaper and more effective. But distance learning is not necessarily more cost effective; it depends on the design of the distance learning course or program. A common approach to online learning is the following. Full-time faculty design and then offer asynchronous courses or asynchronous parts of courses over the Internet. Many faculty find this approach educationally sound but also inordinately time consuming. Because the medium itself permits active participation by all students in every discussion, faculty can feel obliged to respond to dozens of student postings each day. Whatever the learning virtues of this kind of instruction may be, it does not lower per-student costs; it raises them. In many ways, this mode is more costly than traditional classroom delivery. The more one replicates the traditional campus model, the more one's operating costs will resemble or exceed traditional campus costs, as in the case of instructor-led models, such as televised classes or computer conference-based courses, which rely on the same student/faculty contact as traditional models. Similarly, if one uses site-based delivery methods (versus desktop delivery to the home or office), the same borrow-rent-buy facilities issues found on campus will arise. The bottom line Doing a careful analysis of the instructional tasks associated with the traditional course format allows one to gain an understanding of those that can be shifted from personnel to technology-based materials and those that cannot. After determining the pedagogical principles that need to be employed in the redesign and the kinds of instructional personnel who are essential to the specific tasks, one can experiment with a variety of redesigns and calculate their associated costs. Most academic problems can be addressed in a variety of ways; there is no one perfect redesign strategy. The principles are generic, however. Cost savings result from shifting the time spent by instructional personnel to the technology. At the end of the symposium, the participants reconsidered the assumptions listed at the beginning of this paper. They were asked, does anyone still believe that improving quality has to mean increasing costs? No, they said. By employing well-thought-out redesigns, it is possible to improve quality without adding costs. Does anyone believe that adding information technology always adds cost? Again, the answer was the same. And finally they were asked, does anyone still believe that using information technology threatens the quality of higher education? Again, the answer was the same. It depends on the design of the course. The participants agreed strongly that the concept of readiness criteria is an essential one. Large-scale redesign is not a trivial process. It requires a high degree of preparedness in order to be successful. Participants noted, however, that as more institutions go through the process of redesign, those that follow will be the beneficiaries of what the early pioneers have learned. The group also agreed with placing primary emphasis on improving learning when thinking about redesign. Some went so far as to argue that if these new environments provide a better way to deliver education, why base the argument for a cultural change within our college and universities on the fact that it will save money? Others countered that there can be no quality improvements without controlling costs because innovations cannot be sustained without doing so. A wealth of experience shows that attempts to add on innovations with external support, and without internal structural change-especially the commitment of resources in the institution's core budget-have been almost totally unsuccessful. In order to be sustained, changes in instructional practice must be affordable by institutions and integrated into their base funding practices. Since the participants became convinced that it is possible to enhance learning while reducing costs, at the end of the day the discussion returned to one of our starting points: the different views of higher education's stakeholders as to who should harvest the savings. Is it the department or the instructor, the institution or the students? Legislators would prefer to see some, if not all, of the savings passed on to the public or to the consumer in some way, by reducing tuition, for example. If some or all of the savings are retained by the institution, what should be done with them? Should the extra resources be reinvested in the course's ongoing development? Perhaps the academic unit should capture the savings. Or should the savings be returned to the institution to be reallocated for other uses? If the savings are captured by the department or by the institution, there is little incentive for faculty members to improve productivity by increasing enrollment or improving retention. Some believe that the faculty members involved in the redesign should benefit directly as an incentive or a reward for increasing productivity. If the individual instructor captures the savings (faculty time), it may mean more time to do research or it could mean more time to pursue personal interests. How we reward faculty and staff for increased productivity is an important consideration. Once it is possible for institutions to create a surplus of instructional resources rather than simply consuming them, we will be forced to rethink many of our assumptions about planning and budgeting. A whole host of institutional policy issues will be involved as well as numerous practical matters having to do with supporting innovation. What a great topic for a future Pew symposium! Lanny Arvan John Cahir Gordon K. Davies Richard Ekman Ann S. Ferren William H. Graves Robert C. Heterick, Jr. Robert A. Hoover Carolyn Jarmon C. Peter Magrath Lillian McDermott Anne H. Moore John Moore William M. Plater Michele Seidl Charles W. Steger Carol A. Twigg Jack M. Wilson Rapporteurs Patricia Bartscherer Susan Oaks Improving Learning & Reducing Costs: Redesigning Large-Enrollment Courses by Carol A. Twigg ©The Pew Learning and Technology Program 1999 Center for Academic Transformation Rensselaer Polytechnic Institute
|
 |
 |
|||||||||
 |
|
|
|
|
|
|
|
|
|
|
|
|
