Student Evaluations vs. Quality Matters in Online Learning

High quality course design and delivery is important to the ongoing growth and success of the Speed School’s online programs. End-of-term student evaluations provide one way of assessing both online and face-to-face courses across various factors. In addition to asking about departmental learning objectives aligned with ABET accreditation standards, these surveys seek student feedback on the following items.

  1. Were the goals of this course established?
  2. Were the goals of this course met?
  3. To what extent has your knowledge and/or skill been increased as a result of taking this course?
  4. Rate the effectiveness of this instructor’s classroom presentation (e.g., preparedness, delivery, format, use of illustrative examples, etc.).
  5. Rate instructor’s attitude toward students (e.g., availability outside class, adequacy of office hours, adherence to scheduled class times, adherence to syllabus, general attitude toward students, etc.).
  6. Rate the overall effectiveness of this course.
  7. Rate the overall effectiveness of this instructor.

The average ratings for each of these items can be useful when comparing courses across sections, instructors, departments and delivery modes, but they must be used with caution, especially when the number of respondents is low. One of the most helpful aspects of these student evaluations is the written comments, which provide additional insights about outcomes, strengths, areas for improvement and overall satisfaction.

There are several shortcomings of student evaluations, especially when it comes to online learning. These include their backward-looking nature (feedback collected after much or all of a course has been delivered), the expertise of the evaluators (student participants in a course instead of experts in course design and delivery) and their focus on traditional face-to-face delivery (which does not account for the unique demands of effective online courses). In response to these and other concerns ten years ago, a new evaluation tool was developed based on national standards of best practice, research findings and instructional design principles.

As stated in the 2011-2013 Quality Matters Rubric Workbook for Higher Education, “Quality Matters (QM) is a faculty-centered, peer review process that is designed to certify the quality of online and blended courses. QM is a leader in quality assurance for online education and has received national recognition for its peer-based approach and continuous improvement in online education and student learning.” QM subscribers include the University of Louisville’s Delphi Center for Teaching and Learning, as well as hundreds of post-secondary, K-12 and other academic institutions.

At the heart of the Quality Matters toolset and process is the QM Rubric’s eight general standards (see below) and 41 specific review standards, of which 21 are identified as essential (three points each), 12 as very important (two points each) and eight as important (one point each). In order for a UofL online course to be “Delphi Certified” using the QM Rubric, it must meet all 21 essential standards and achieve a total score of at least 81 out of 95 possible points.

  1. Course Overview and Introduction
  2. Learning Objectives (Competencies)
  3. Assessment and Measurement
  4. Instructional Materials
  5. Learner Interaction and Engagement
  6. Course Technology
  7. Learner Support
  8. Accessibility

Quality Matters is an important element of the Delphi Center’s training of current and future online faculty. This spring, eight Speed School instructors participated in the Survivor’s Guide to Online Teaching and Learning, a 12-hour program over four weeks in the Vogt Building. Congratulations and thank you to Dr. Antonio Badia (CECS), Dr. Ahmed Desoky (CECS), Mr. Gary Eisenmenger (ENGR), Dr. Jeff Hieb (ENGR), Dr. Ibrahim Imam (CECS), Dr. Anup Kumar (CECS), Mr. Michael Losavio (CECS) and Dr. Olfa Nasraoui (CECS) for their attendance and participation. There are seven more Speed School faculty who will be completing a one-week face-to-face or eight-week online session of Delphi U this summer. All of these instructors have received or will receive QM training that will help them develop a new online course or improve an existing one. After teaching the new or improved course once or twice, all will be encouraged to seek a formal Quality Matters review, which we hope will eventually lead to a “Delphi Certified” designation for all of Speed’s online courses.

Moving forward, we will rely on both Quality Matters and student course evaluations as we strive for continuous improvement in online teaching and learning. QM emphasizes the use of research-based best practices for course design using a faculty-driven, peer review process, whereas student course evaluations focus on effective curriculum delivery and achievement of desired learning outcomes. Responding positively to feedback from both assessments will help Speed School faculty, staff and students to design, deliver and benefit from exceptional online learning experiences.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s