Assessing Teaching Effectiveness Committee
For several decades, universities and colleges have attempted to measure the quality of teaching by relying on student evaluations. This sole means of evaluation is convenient and cheap to administer. It provides tangible measures that allow universities to mention it and evaluate the faculty member. In 2009, the Faculty Union at Ryerson University in Toronto filed a grievance with the University over teaching evaluations being an unfair measure of teaching effectiveness. The arbitrator ruled in favor of the faculty and noted that student evaluations can no longer be used for high stakes decisions such as tenure and promotion. Most traditional student evaluation surveys include broad questions such as the quality of the course overall and quality of the instructor overall. Such measures have been criticized as they tend to garner biased responses.
Several studies clearly show bias in student opinion surveys (examples, MacNeil et al 2015; Mitchell 2018; Peterson et al 2019; Physics Today 2020; Weinkle et al 2020; Kreitzer & Sweet-Cushman 2021). There is both measurement bias and equity bias in student evaluations of teaching. Measurement bias partially emanates from the lack of attention to the measures as being about perceptions, opinions, or actual behavior/experience. The course and student characteristics involved in measurement bias include generally lower evaluation scores in large and required classes and lower workload; disciplinary differences – humanities high and natural sciences low’ and lack of correlation between teaching and learning. Equity bias is rooted in the gender, race, accent of the instructor; among other differences including body ability. Gender bias in student evaluations of teaching is strong based on the evaluator and written comments typically display very strong gender bias (Kreitzer & Sweet-Cushman 2021). While racial biases are also evident, this aspect has been less studied.
A few institutions (examples – Georgia Institute of Technology; Purdue; UMass, Amherst; USC) have revisited and revised teaching evaluations by focusing on effectiveness and adopting a more holistic approach. There is an urgent need for VCU to reconsider its approach to assessment of teaching.
While student evaluations of teaching can be an important tool for evaluating and improving teaching and learning, there is substantial peer-reviewed evidence from other colleges and universities that student course evaluations of the sort used at VCU are biased with respect to gender and race and that the numerical scores maybe poorly correlated with teaching effectiveness and learning outcomes. There is also scholarly evidence that peer (faculty) reviews of teaching may be ineffective at evaluating teaching effectiveness and giving useful feedback for improvement.
References
Kreitzer, Rebecca J. & Jennie Sweet-Cushman. 2021. “Evaluating Student Evaluations of Teaching: a Review of Measurement and Equity Bias in SETs and Recommendations for Ethical Reform.” Journal of Academic Ethics 20 (1):73-84.
MacNeil, Lillian, Adam Driscoll & Andrea N. Hunt. 2015. “What’s in a Name: Exposing Gender Bias in Student Ratings of Teaching.” Innovative Higher Education 40: 291–303.
Mitchell, David. 2018. The Ecology of Inclusive Education: Strategies to Tackle the Crisis in Educating Diverse Learners. London, Routledge.
Peterson, D.A.M., L.A. Biederman, D. Anderson, T.M. Ditonto, K. Roe, “Mitigating Gender Bias in Student Evaluations of Teaching. PLoS ONE, 14(5), May 15, 2019. https://doi.org/10.1371/journal.pone.0216241
Physics Today. “Reevaluating Teacher Evaluations in Higher Education.” January 2020
VCU Faculty Senate Report. White Paper on Evaluating Teaching Effectiveness. Academic and Professional Status Committee of VCU Faculty Senate - Kristine Artello, Chair and Masoud Manjili, Co-Chair
Weinkle, Laura J., Jennifer M. Stratford, Lisa M. J. Lee. 2020. Research Report -Voice in Digital Education: The Impact of Instructor's Perceived Age and Gender on Student Learning and Evaluation. Anatomical Sciences Education, 13(1): 59-70.