Orientation for Speed Online Students

A Blackboard organization named Speed Online has been launched to help incoming online graduate students prepare for personal, academic and career success. With new students enrolling in five fully online graduate degree and certificate programs offered by three academic departments, the new organization will simplify and expedite the orientation and enrollment process for online graduate students.

Shortly after applicants are offered admission, they are added to the Speed Online organization and emailed instructions for logging in to Blackboard. Outlined below, the main menu not only guides them through the orientation process, but also provides 24/7 access to the information, resources and services they may need throughout their program.

• Welcome: Orientation overview and welcome message from Dean Pinto.
• New Students Start Here: Initial action required of newly admitted students, including account setup, course registration, bill payment and financial aid.
• Speed Online 101: General information about technology, textbooks, advising, academic integrity, policies and student expectations.
• UofL Resources: Details about helpful on-campus and online resources students are most likely to need.
• Announcements: Occasional messages posted for all Speed Online students.
• Discussion Board: Community forum for questions, comments and responses of general interest to all students in any of our online programs or courses.
• Faculty & Staff: Names and contact information for online program directors and school administrators.
• Calendar: Important academic dates and deadlines for the university, school and online programs.

Engineering Management, Computer Science and Data Mining students enrolling for the first time in Fall 2014 are the first ones to use the Speed Online orientation process. To give new students more hands-on experience with Blackboard tools used in online classes and to help them feel more connected to the Speed School community, we are planning two additional modules for those enrolling in Spring 2015.

• Pre-Term Assignments: Confirm technology setup, participate in Blackboard Collaborate meeting, develop online learning plan, introduce self to faculty and students using a blog, expand LinkedIn professional network and complete pre-enrollment quiz.
• Orientation Programs: Required synchronous or asynchronous virtual orientation session for Speed Online students, as well as optional on-campus orientation at the School of Interdisciplinary and Graduate Studies (SIGS) and reception at the Speed School.

The new Speed Online organization has three main benefits. First, it will ensure that all incoming online students have timely access to high quality information that is relevant, accurate and complete. Second, it will help improve enrollment, retention and graduation rates for online programs and increase the engagement of online students and alumni in the Speed School community. Finally, it will reduce the number of times that academic departments, program directors and online instructors have to answer frequently asked questions about policies, procedures, resources and technology. Please contact Todd Reale, Director of Online and External Programs, at todd.reale@louisville.edu or 502-852-5012 if you have any questions, comments or suggestions.

 

Student Evaluations vs. Quality Matters in Online Learning

High quality course design and delivery is important to the ongoing growth and success of the Speed School’s online programs. End-of-term student evaluations provide one way of assessing both online and face-to-face courses across various factors. In addition to asking about departmental learning objectives aligned with ABET accreditation standards, these surveys seek student feedback on the following items.

1. Were the goals of this course established?
2. Were the goals of this course met?
3. To what extent has your knowledge and/or skill been increased as a result of taking this course?
4. Rate the effectiveness of this instructor’s classroom presentation (e.g., preparedness, delivery, format, use of illustrative examples, etc.).
5. Rate instructor’s attitude toward students (e.g., availability outside class, adequacy of office hours, adherence to scheduled class times, adherence to syllabus, general attitude toward students, etc.).
6. Rate the overall effectiveness of this course.
7. Rate the overall effectiveness of this instructor.

The average ratings for each of these items can be useful when comparing courses across sections, instructors, departments and delivery modes, but they must be used with caution, especially when the number of respondents is low. One of the most helpful aspects of these student evaluations is the written comments, which provide additional insights about outcomes, strengths, areas for improvement and overall satisfaction.

There are several shortcomings of student evaluations, especially when it comes to online learning. These include their backward-looking nature (feedback collected after much or all of a course has been delivered), the expertise of the evaluators (student participants in a course instead of experts in course design and delivery) and their focus on traditional face-to-face delivery (which does not account for the unique demands of effective online courses). In response to these and other concerns ten years ago, a new evaluation tool was developed based on national standards of best practice, research findings and instructional design principles.

As stated in the 2011-2013 Quality Matters Rubric Workbook for Higher Education, “Quality Matters (QM) is a faculty-centered, peer review process that is designed to certify the quality of online and blended courses. QM is a leader in quality assurance for online education and has received national recognition for its peer-based approach and continuous improvement in online education and student learning.” QM subscribers include the University of Louisville’s Delphi Center for Teaching and Learning, as well as hundreds of post-secondary, K-12 and other academic institutions.

At the heart of the Quality Matters toolset and process is the QM Rubric’s eight general standards (see below) and 41 specific review standards, of which 21 are identified as essential (three points each), 12 as very important (two points each) and eight as important (one point each). In order for a UofL online course to be “Delphi Certified” using the QM Rubric, it must meet all 21 essential standards and achieve a total score of at least 81 out of 95 possible points.

1. Course Overview and Introduction
2. Learning Objectives (Competencies)
3. Assessment and Measurement
4. Instructional Materials
5. Learner Interaction and Engagement
6. Course Technology
7. Learner Support
8. Accessibility

Quality Matters is an important element of the Delphi Center’s training of current and future online faculty. This spring, eight Speed School instructors participated in the Survivor’s Guide to Online Teaching and Learning, a 12-hour program over four weeks in the Vogt Building. Congratulations and thank you to Dr. Antonio Badia (CECS), Dr. Ahmed Desoky (CECS), Mr. Gary Eisenmenger (ENGR), Dr. Jeff Hieb (ENGR), Dr. Ibrahim Imam (CECS), Dr. Anup Kumar (CECS), Mr. Michael Losavio (CECS) and Dr. Olfa Nasraoui (CECS) for their attendance and participation. There are seven more Speed School faculty who will be completing a one-week face-to-face or eight-week online session of Delphi U this summer. All of these instructors have received or will receive QM training that will help them develop a new online course or improve an existing one. After teaching the new or improved course once or twice, all will be encouraged to seek a formal Quality Matters review, which we hope will eventually lead to a “Delphi Certified” designation for all of Speed’s online courses.

Moving forward, we will rely on both Quality Matters and student course evaluations as we strive for continuous improvement in online teaching and learning. QM emphasizes the use of research-based best practices for course design using a faculty-driven, peer review process, whereas student course evaluations focus on effective curriculum delivery and achievement of desired learning outcomes. Responding positively to feedback from both assessments will help Speed School faculty, staff and students to design, deliver and benefit from exceptional online learning experiences.

Online Learning Improvement Plan

On Friday, February 28, Speed School Dean Neville Pinto sent the following message to engineering faculty and staff. It connects the school’s strategic plan with recent course evaluations to create a call for action to improve the design, development, delivery and assessment of Speed’s online courses and programs. We welcome your feedback and invite you to share your experiences with online learning, suggestions for improvement and ideas for potential new programs. Please contact me directly at todd.reale@louisville.edu or 502-852-5012.

---

A core element of the Speed School’s mission is to serve our university, community and profession by providing high quality engineering education programs. To realize our vision of becoming the preeminent provider of high quality, accessible and experiential-based engineering education, we must strive for excellence in all that we do. This includes not only attracting a greater diversity of motivated, prepared and talented students into all degree programs, but also engaging students with state-of-the-art engineering curricula and enhanced modes of delivery in our classrooms, labs and online.

We have analyzed course evaluations from 19,793 students enrolled in 1,604 sections of 363 courses taught by 177 instructors in eight departments from Fall 2011 through Fall 2013. While there are many reasons to celebrate and be proud of our departmental and individual accomplishments in both face-to-face and virtual settings, we must strive to close the gap between the two. In general, online courses are not as highly rated by students, particularly in the areas of instructor preparedness, content delivery, course format, use of illustrative examples and overall effectiveness. Students have also reported concerns about the quality and variety of course materials, appropriate use of learning technologies, communication with instructors and interaction with fellow students, as well as responsiveness to program inquiries and admission applications by prospective students.

Despite these general shortcomings, we know that online courses can be equally effective and successful compared to those delivered in the classroom. In its first semester last fall, our new online engineering management program outperformed the averages for all eight quantitative measures when compared not only to all other Speed online courses, but also to all Speed classroom-based courses. Moving forward, we are working with the university’s Delphi Center for Teaching and Learning to develop and implement plans that will help us improve the quality and consistency of both face-to-face and online courses.

• Preparation: If they have not done so already, faculty planning to teach online courses should participate in specialized Delphi Center training. Those teaching this summer are asked to attend Survivor’s Guide training at the Speed School on four Fridays this spring. Those scheduled for next fall or spring will be asked to complete Delphi U prior to teaching online. Additional details will be communicated directly to faculty through the Office of Academic Affairs later this week.
• Improvement: As part of the above training, current and future online instructors will be educated about Quality Matters, a faculty-centered peer review process that is designed to certify the quality of online and blended courses. The QM Rubric will help us improve course design by assessing 41 clearly defined review standards and providing substantial, constructive and specific comments about course strengths and suggestions for improvement. At least eleven existing Speed online courses will be reviewed over the coming year and all new Speed online courses and instructors will be evaluated after one or two future offerings.
• Development: Delphi Center staff will support us as we continue to develop, improve and expand our online offerings. Among other things, they will assist with faculty development; teaching with technology; instructional design; online program development, marketing, recruiting and student services; and Blackboard and other technology support services. They are a valuable on-campus resource for anyone wishing to improve teaching and learning in their classrooms or online.
• Recognition: Two years ago, U.S. News and World Report started its annual ranking of Best Online Graduate Engineering Programs, which provides another opportunity for us to promote our growing online degree and certificate programs. Although the university decided not to participate while the methodology was still being developed in the first year, the Speed School was listed, but not ranked, in the most recent 2014 survey. Student engagement (30%), faculty credentials and training (25%) and student services and technology (20%) account for most of each school’s ranking. These broad categories include smaller factors, such as the presence of tenured and tenure-track faculty with terminal degrees, online teaching best practices training, certified instructional designers, technological infrastructure, collaborative coursework, required course evaluations and more. The above plans for training and assessment will help us communicate a more favorable message about the quality and value of a Speed School graduate degree or certificate.

In conclusion, there are many benefits to embarking on this plan to improve our online programs, including clearer standards, improved teaching quality, enhanced learning outcomes, expanded online offerings, more student enrollments, increased revenues for the school and academic departments, broader recognition, higher rankings and more. It is my hope that you will join me in adopting, supporting and participating in these new initiatives so that we can accomplish our goals, fulfill our mission and realize our vision together.