Download as PDF (Introduction and Item Bank)
Student ratings of teaching
effectiveness have been researched for over 50 years and are the subject of over 2,500 books
and articles. They are the most widely studied of all forms of teaching evaluation and the
most supported by empirical research. This extensive work clearly demonstrates that properly
designed student rating instruments can provide feedback that is valid and reliable,
relatively free from bias, correlates well with external measures of student learning, and
serves multiple purposes.
Student ratings of teaching effectiveness are a valuable source of information and can
guide efforts for improving teaching. They are, however, only one source of data for
assessing teaching effectiveness and most useful in conjunction with a more comprehensive
evaluation process. Evaluation specialists recommend considering multiple sources of data
(e.g., student ratings, self-assessment, classroom observations, peer review of materials,
teaching portfolios, alumni ratings) for both improvement purposes and personnel decisions.
What Is Effective Teaching?
The development of the OIRA item bank began with a review of the literature from several
areas related to student ratings of teaching effectiveness. This encompassed research on
effective teaching, learning, principles of good teaching practice, evaluation, and student
ratings. The culmination of this analysis resulted in the creation of the OIRA item bank
that is broad based and reflects the currents ideas and work of leading researchers in these
fields.
Teaching and Learning Literature
The extensive literature in the areas of teaching and learning provided the framework for
The Seven Principles of Good Practice in Undergraduate Education (Chickering and
Gamson). Originally published in 1987, the principles were developed by a distinguished
group of researchers and commentators on higher education. The principles assert that
several common features characterize good practice in undergraduate teaching.
- Encourages student-faculty contact is evident through interaction and rapport
with students, acceptance of students’ view and discussions, concern with student
progress, and willingness to help students with problems.
- Encourages cooperation among students occurs through the active involvement of
students in small groups activities.
- Encourages active learning is addressed through talking and writing about learning,
relating learning to past experiences, applying learning to present events, and projecting
learning into the future.
- Gives prompt feedback is accomplished by providing timely feedback that is supportive and
appropriate for enhancing improvement. The quality of the assessment tool (e.g., exam,
paper, oral presentation) and the frequency of feedback are also important components.
-
Emphasizes time on task is evident through effective use of class time, appropriate time
allocation for course components, enhancement of students’ time management skills, and
course pacing.
- Communicates high expectations is reflected in setting high but attainable goals for
students’ academic performance, difficulty of coursework, and course workload.
- Respects diverse talents and ways of learning
encompasses recognizing different talents
and learning styles of students, being sensitive to differences among students, and
recognizing unique interests and talents of students.
Evaluation and Student Ratings Literature
Several meta-analysis studies from the evaluation and student ratings literature reviewed
research in which students, faculty, administrators, and alumni identified components of
effective teaching (Feldman). Results showed considerable overlap between the qualities
selected by each group. The studies defined common characteristics of effective teaching.
- Appropriate student learning outcomes place the focus on the outcomes of effective
teaching. They can be classified by a variety of schemes including cognitive vs. affective,
or low-level (i.e., acquisition of knowledge) vs. high-level
(i.e., analysis). It is important to ensure that student learning is consistent with course
objectives.
- Flexibility in teaching approaches permits faculty to enhance student learning by
responding to diverse student backgrounds and learning styles. Research indicates that
engaging students in the learning process contributes to their cognitive development. It has
also been documented that faculty who implement a variety of appropriate instructional
approaches are more enthusiastic in their teaching.
- Good organization and preparation of course is reflected in the level of preparation, use
of class time, and in the course objectives, syllabus, assignments, activities, and
evaluation methods of student performance. Organizational strategies are important as
research indicates they are related to how much students learn.
- Knowledge of and enthusiasm for the subject matter and teaching should be evident.
Faculty must be knowledgeable in their subject matter in order to organize it in a
meaningful way for students. They should be able to communicate their knowledge at a level
students can comprehend. An infectious enthusiasm comes with confidence and excitement for
the subject and teaching.
- Effective communication is important in all types of educational settings from large
lectures to one-on-one conversations. It is manifested through good speaking, writing, and
listening skills. Communication also occurs in a variety of other ways including a course
syllabus, presentation, explanation, or course activities.
- Positive attitude toward students is evident in a teaching environment that supports
student learning and provides a positive self-image for students. It is reflected in mutual
respect and rapport, concern for students’ learning, availability to students outside of
class, and encouragement of students to participate in discussion and express their
opinions.
- Fairness in evaluation and grading is reflected in the consistency between course
objectives, course content, assignments, and evaluation strategies. The standards of grading
should be clear and consistent, feedback timely and useful, and the equity of the workload
appropriate for the credits received.
Organization of the OIRA Item Bank
Research indicates that effective teaching is a complex, multidimensional process that
should be reflected in the design of student rating instruments. Consequently, most
published student rating forms have a well-defined factor structure and provide measures of
distinctive components of teaching effectiveness. The organizational scheme of the OIRA item
bank was modeled on this work.
The item bank was developed to guide faculty in designing student rating forms that reflect
the questions they would like to have answered with respect to their own teaching. With this
collection of items, faculty, departments, and schools/colleges can customize their forms in
the areas of student learning, teaching practice, and course elements.
The item bank contains questions organized around various dimensions of effective teaching
and special instructional settings. The items were categorized on the basis of their
apparent content. The dimensions include:
Student Learning
- student outcomes (cognitive, critical thinking, interests and values, social awareness,
self-concept)
- collaborative/cooperative learning
- student effort and involvement
Teaching Practice
- organization and preparation
- communication
- faculty/student interaction
Course Elements
- grading
- examinations
- textbook
- assignments (reading, writing, oral presentation, computer-based, other)
- audiovisual aids
- technology usage
- course difficulty, pace, and work load
Overall (Global) Questions
Student Demographic Information
Open-ended Questions
Specific Activities
- guest speakers
- field trips
- research and field projects
Special Instructional Settings
- team teaching
- laboratory sessions
- discussion/recitation sections
- studio work
- community-based and service learning
- clinical/field placements
- graduate seminar
- research supervision
The selection of a student response scale for items within the OIRA item bank is based on
a review of the research and examination of nationally published instruments. Items are
written so student responses are recorded on a 5-point agreement scale with:
1 = strongly disagree
2 = disagree
3 = neutral (neither disagree nor agree)
4 = agree
5 = strongly agree
Open-ended questions asking for written responses from students are also included in the
item bank.
Creating Your Student Rating Form
Step 1: Determine the purpose of the instrument
Historically, student ratings of college teaching were designed for purposes of improving
teaching practice or performance (i.e., formative evaluation). Over time, however, student
ratings have increasingly been used to assist in personnel decisions (i.e., summative
evaluation). In too many cases, this has become their exclusive purpose. Evaluation that
serves both purposes is possible, however, research suggests that it is preferable to
separate formative and summative evaluation – both conceptually and in practice.
Furthermore, guidance for those who use student ratings of instruction is important in order
to avoid misuse and misinterpretation of the results.
Step 2: Select the appropriate teaching dimensions
National studies suggest that the areas of student learning, teaching practice, and
course elements be included on student rating forms. A comprehensive instrument would
reflect the following:
- student outcomes
- student effort and involvement
- organization and preparation
- communication
- faculty/student interaction
- assignments, exams, and grading
- course difficulty, pace, and work load
- overall (global) questions
- student demographic information
Step 3: Choose the specific questions
The number of questions you select from each of the areas you identified in step two will
determine the length of your form. Three of the leading national surveys (SIR II, SEEQ,
IDEA) contain 40 to 45 questions, but you may choose the number of questions that you find
appropriate for your purposes. You may also want to make selections from the more
specialized sets of questions within the item bank. These may be appropriate if you have
specific activities or instructional settings that you would like to assess. Open-ended
response items are a final option for your student rating form. Generally, two or three
open-ended questions are appropriate. Possible selections are available in the item bank.
Step 4: Organize the selected items
Consideration should be given to: How will the items be grouped (e.g., clustered by
teaching dimension, chronologically)? Will labels be used to identify the subtitles (e.g.,
student outcomes, communication)? How will the items be arranged for ease of reading and
answering?
(View Item Bank Questions)
top
More Information
If you would like more information on this topic, please contact the
Office by email or by phone at 443-8700.
