Improving the Quality of Student Learning
Based on available data about learning outcomes from the course pilot, what were the impacts of re-design on learning and student development?
The redesigned course ran during the fall 1999 and spring 2000 semesters. The enrollments at the end of the course were 1406 and 639, respectively, and the final grade averages were 2.22 on a scale of 4 in both cases. Enrollment in the fall semester of 2000 was about 1360; outcome data are not yet available.
These classes were taught in one section each in the redesigned format. The last year for the traditional course was 1996 - 1997; a preliminary form of the redesigned format was introduced to all sections in 1997 - 1998, the year the Math Emporium was opened. The first major revision of the on-line presentations was 80% complete for fall 2000, and the remainder will be in place for spring, 2001. The goals of these revisions have been (i) to improve clarity in areas where test results show students have difficulty, and (ii) to make the presentations substantially more interactive.
The main assessment data come from the final exam and course grades. In addition, opinion surveys, student focus groups and interviews with staff are used. (Note: Honors sections of Linear Algebra are not part of the course redesign. They accounted for approximately 5.6% of the students in each semester. Unless otherwise stated, statistics here and below exclude the honors sections.)
All students take a common final exam in multiple-choice format. Questions are coded according to a list of course goals and difficulty levels, and a two-person team reviews the results each semester. The team decides whether each goal covered on the exam has been "well met" (W), "met" (M), or "not met" (N), and the report is used by the course designers as a guide for changes in the syllabus or materials. For fall 1999, the questions yielded 15 W, 6 M and 4 N outcomes. For spring 2000, the results were 13 W, 6 M, and 6 N. In both cases, the reviewers concluded that the overall scores (67% and 69% correct for Fall and Spring, respectively) were reasonable for these tests.
These and earlier final exams have consistently found lower levels of success in the topic of eigenvalues and eigenvectors than in the course as a whole. This is one of the more complex topics of the course, and it is covered near the end of the term. This area is one focus of attention for the revised presentations. At this point, there is a consensus that the system works best with simpler, skills-oriented topics than with more conceptual ones.
Course enrollment and grade statistics have been steady from year to year and consistent with those of the last traditional year. The table below shows data for fall semesters:
(These numbers include the honors sections.) The general conclusion is that the revised course has yielded roughly stable performance, with some increase in retention.
There is a "blip" in grades for fall 1998. Various faculty members who have been involved with the course feel that they were perhaps more accommodating to the students during the first two years of redesign. For example, early software problems led to numerous disputes over quiz grades, and these were generally resolved in favor of the student.
For the spring 2000 semester, the exam reviewers looked closely at test results for students who were repeating the course. They did considerably worse than the group at large, accounting for 11% of the spring enrollment and 40% of the exam scores below 50. This indicates, perhaps, that the various learning paths offered in the revised course still do not reach everyone.
We have not identified particular advantages or disadvantages for minorities or women in the new format. The class, made up primarily of engineering students, is fairly homogeneous socially.
Assessment also includes surveys of student opinion. Web-based surveys were given in Linear Algebra at the beginning and end of the fall semesters, 1999 and 2000. The surveys are long (e.g., there are 80 questions on the late fall survey), and participation rates are over 90%. A number of interesting points have emerged so far, and the latest results allow us to begin looking at year-to-year progress. [In these examples, wording such as "agree" is shorthand for survey categories Agree or Strongly Agree, Frequently or Always, etc. Percentages here and below are relative to all respondents to the survey, including those who did not respond to the particular question.]
Program in Course Redesign Quick Links: