Download as PDF
Data collected from student ratings of teaching effectiveness may be from forced-choice questions, open-ended inquiries, or a combination of these. Careful interpretation of the data in combination with reflective practices will provide you with valuable information about your teaching effectiveness.
Common Statistical Terms – What they mean and how to use them
The following are common statistical terms often found on data analysis reports.
The letter “N” represents the sample size (number of students who responded overall or to a particular item).
- It is important to compare the number of student responses (N) to the total class enrollment. Additionally, check the N for each question since students may not answer every item. Results should be interpreted with caution if less than 75% of the enrolled students completed the survey.
- Reliability (i.e., consistency) of the data is also compromised with a sample size of fewer than ten students.
The mean score represents the numerical average for a set of responses. The following points assume a scale in which a low score is assigned to negative responses (i.e., poor) and a high score to positive responses (i.e., excellent).
- Generally, the higher the mean score, the better the evaluation.
- It may be helpful to rank order the mean scores for a set of evaluations in order to identify those that are higher or lower than the majority of your scores. For example, on a 5-point scale, if most of your mean scores tend to cluster around 3.8, then those falling below that value (approximately 3.4 or lower) may signal areas of teaching that need attention.
- On a 5-point scale, items with mean scores above 4.0 generally reflect teaching aspects that are particularly effective.
The standard deviation represents the distribution of the responses around the mean. It indicates the degree of consistency among student responses.
- The standard deviation is often abbreviated in data tables as s, sd, SD, std, or StD.
- The standard deviation in conjunction with the mean provides a better understanding of your data. Begin by adding the standard deviation to the mean. Next subtract the standard deviation from the mean. The range between the two calculated values represents where approximately 2/3 of your students’ responses fall. For example, if the mean score is 3.3 with a std of 0.4, then 2/3 of the students’ responses lie between 2.9 (3.3 – 0.4) and 3.7 (3.3 + 0.4).
- The standard deviation represents the degree of similarity among the students’ responses. A small standard deviation (as in the example above) reflects a high degree of consensus among the students. Since there is a small numerical range (2.9 – 3.7) in which 2/3 of the ratings fall, the response pattern among the students is very consistent.
- A large standard deviation indicates that there was considerable disagreement among the students’ responses. For example, if the mean score is 3.3 with a std of 1.0, then 2/3 of the students’ responses lie between 2.3 and 4.3. This indicates a wide disparity among the responses to this item, with the mean simply representing a numerical average of the responses and not a consensus rating by the class. In such a situation, carefully examine the survey question. Might a particular activity or teaching approach not be equally effective for all students? Perhaps the class is not in agreement on course organization or grading. Why might this be? How can you make adjustments to your teaching that would more readily meet the needs of all students?
Students’ Written Comments – What they mean and how to use them
Including open-ended questions on student rating forms can help to clarify statistical data and provide valuable suggestions for course improvement. By asking students to write comments and allowing sufficient time for completion of the evaluation, you can increase the number and quality of written comments.
Making Sense of the Comments
It is often difficult to interpret written comments because they lack the built-in structure found in the forced-choice component of the evaluation. Initially, the comments appear to be a collection of random, unconnected statements. Bringing a sense of organization to the written comments can provide important insights into your teaching.
Strategies for Organizing Student Comments
- Read, organize, and compare the comments from each question separately. That is, if you have more than one open-ended question on your evaluation, begin by reading the responses to just the first question on all the evaluations. Then go back and read the responses to the second question, then the third question, etc. This approach permits you to focus on one topic at a time.
- Group the comments by teaching components (i.e., course organization, communication, faculty/student interaction, feedback) or by categories that are most meaningful to you. The grouping of comments by similar topics allows you to focus on comments related to one teaching component at a time.
- Depending on how your data were analyzed and the type of student information collected, you may be able to correlate the students’ comments with items such as: their major, year of study, expected grade, and overall instructor and/or course rating. This type of analysis may provide insight into how various subpopulations of students view the course.
- Once the students’ comments have been organized and analyzed, compare them with your quantitative feedback. Look to the written comments for explanations of the ratings on the forced-choice responses. For example, students may have given lower overall ratings to the questions addressing “faculty/student interaction.” Written comments may reveal consistent student concerns about your helpfulness and availability outside of class. By exploring this link between the responses to open-ended and forced-choice questions, you may be able to identify specific changes that would enhance your teaching effectiveness.
Positive and Negative Comments
- Research indicates that the students who are very satisfied or very dissatisfied generally provide written comments.
- It is helpful to determine the proportion of negative to positive comments for interpretative purposes. This will assist you in determining if the comments are representative of the entire class or a small minority of students.
- Comments that reflect positively on your teaching effectiveness can usually be considered genuine. Since the course evaluation is anonymous, students do not usually write positive comments unless they mean them.
- Since students’ anonymity is protected on student ratings, students may write negative comments that range from sarcastic to vicious. Obviously, not all of these comments are constructive. Pressures unrelated to you or your course may also underlie some of these comments. Keeping this in mind may help to limit overreaction to certain comments.
- Negative comments need to be interpreted with caution. Normal human behavior often causes one to take the negative comments to heart, regardless of how small the number.
- Reference any questions from the forced-choice portion of the evaluation that relate to the issues raised in the negative comments. If these questions received lower ratings, the negative comments may reflect a concern with that particular dimension of your teaching.
Inconsistent or Contradictory Comments
When students’ comments primarily reflect satisfaction (or dissatisfaction) with various aspects of your teaching effectiveness, you can be fairly confident in your response. Occasionally, student comments can be contradictory. For example, one subset of students may indicate they found the interactive group activities enhanced their learning. Another group of students may have found the same activities a waste of their time. These inconsistencies are often due to variations in student development and/or preferred student learning style.
Large introductory level classes, which often attract students with a wide range of developmental levels, may be especially prone to such inconsistencies. There may be students who are not yet developmentally capable of accepting the challenges of your course. They may not be comfortable thinking independently, accepting a high degree of individual responsibility, or reasoning at higher cognitive levels. If this is the case, develop strategies to help these students succeed in your course.
Another explanation for contradictory comments may be that students are not familiar or comfortable with a particular teaching method or assignment model. Students who prefer a teacher-centered, theoretical-based learning environment may be uncomfortable with an
inquiry-based approach that requires group discussions and student interaction. If learning style preferences appear to be the source of the discrepancy, help students to understand the advantages of a particular teaching method and to expand their learning style repertoire. Additionally, varying the teaching style that you use will accommodate the range of learning styles represented among the students in your class.
As one of your final exercises in analyzing your student rating data, take time to step back and reflect generally over the collective data. By addressing the following questions, or others you feel are appropriate, you can begin to view the data from a perspective that allows for insights that will positively impact your teaching effectiveness.
- Is the overall evaluation of the course relatively consistent, or are there clear differences in how students are experiencing the course? What may account for these differences?
- What overall responses stand out? Why? How representative of the total response group are they?
- How do the students’ responses compare with the other types of feedback you already have (i.e., nonverbal cues, student questions or concerns raised either in class or during office hours, attendance patterns)?
- What are you doing in the classroom that would result in the type of feedback you received? For example, if students feel that their contributions in class discussions are valued, think what you are doing to encourage them. Conversely, if students express a problem with knowing when assignments and exams are due, reflect on how you communicate these expectations to your students.
- What aspects of your teaching appear to be effective?
- What areas of your teaching could be improved?
- What new understandings/insights have you gained about your teaching?
- Overall, does the pattern of responses tell a story? That is, are there underlying factors affecting scores in a number of areas, or are strengths and weaknesses specific?
- Does your assessment of your teaching match that of your students? If not, why not?
- Which aspects of your teaching do you want to work on improving? How will you go about this? What resources are available to you?
If you would like more information on this topic, please contact the Office by email or by phone at 443-8700.