May 10, 2016
“Mr. Huidekoper, I think you should relax. Try Quaaludes before class.”
(From a student survey)
It is great to hear of more efforts to welcome the
students’ voice in evaluating a teacher’s effectiveness. I celebrate such efforts here (after I first reveal
what students said of me: oh how those
words can sting!). Maybe we will soon find ways to incorporate feedback
from student surveys into the School Performance Framework. Yes—a portrait of a school based in part on
how students view the quality of their classes and teachers. What a concept!
You know the refrain: we should not judge students, or schools,
on one test. We must have multiple measures.
Of course. Equally true in
assessing a teacher’s performance. In
AV#145 I celebrated the evaluation and support that comes from colleagues, as seems possible in the Teacher
Leadership Collaborative Model in Denver Public Schools. If peer review is the best way to improve
classroom instruction, student surveys are often the second best tool—well ahead of the principal’s observation.
Most of what follows comes from national studies (including
references to work in Colorado, especially in Denver Public Schools and the Thompson
School District)—key findings on, and examples of, student surveys playing a
greater role in accountability and growth.
I hope it can be useful resource for such efforts in our state. I make no proposals on how a school,
district, or the Colorado Department of Education might incorporate surveys
into “educator effectiveness”; in my view, the less prescriptive the
better. Let the school own it. I simply argue: handled well, such surveys
can make us better teachers.
But first I begin with a confession: what my students told
me.
In 18 years of teaching – asking students from grades 6 – 12 to
let you know….[1]
The year comes to a close, and the teacher passes out a
survey to his students—with some trepidation. By early May you have spent over
160 days together this year, you have a sense of what has and has not gone
well, you know you never made a great connection with __, __, and __ —and you
recall past surveys where the students’ comments hurt, so you brace for a few
stingers.
Still, you know you must get their feedback. Hey, you’ve been grading them all year—isn’t it their
turn to give you that C-? More importantly, they know how well you
teach better than the principal does. Yes, I mean that—at least for grades 7-12. While you take some comments with a grain of
salt—perhaps that student is ticked off by a recent low score on a paper, or
feels you snapped too harshly at his or her off-task behavior yesterday—your
class knows if your instruction has been clear, energetic, challenging…. Students know your character pretty well too:
every day they assess if you are kind, patient, fair—and if—no, when—you fall short. They know—as you
do, as should your principal—that you can perform well enough for the
administrator’s “walkthrough,” but what has taken place day in day out, beginning back in mid-August … that was no act. What “the kids” will tell you is likely to be
a good reality check.
What do I mean by hurt?
No one will ever again hire this 66-year-old, so I might as well confess:
Response to: “When
speaking about my work, the teacher did not embarrass me.”
One year the response from my two 8th classes
was positive, but almost half of the 7th graders scored me a 1 (strongly disagree) or 2 (disagree). In sharing my results with
the school administration, I wrote: “I believe strongly in using student work
for comment – for pointing our problems, and of course for giving praise, which
I try to do as much as criticize – but these comments from 7th
graders are troubling. I obviously need to address this issue when I meet next
year’s 8th graders in August.”
Response to: “The
teacher did not lecture, or give uninterrupted talks, too often.”
One year, 20 out of 38 7th graders disagreed, as
did 16 out of 35 8th graders.
Extremely worrisome, especially as I
say I emphasize class discussion and participation, and like to think I invite student voice—and
listen well. Tough to swallow, but I
needed to hear it.
Response to: “There
was sufficient variety in the kind of classroom activities that took place.”
In one 7th grade class, 8 of 43 scored me a 2;
with 8th graders, one scored me a 1 and 12 a 2. Another area that needed work.
After spending a year together, I would love to get all
3’s (agree) and 4’s (strongly agree) in response to: “The teacher made an effort to know me
personally and to teach me as an individual.”
But the response one year was discouraging. Most scored me
a 3 or 4, but in 7th grade 9.5 disagreed, and in 8th
grade, 14 disagreed. Not good. In a
small K-8 school, where classes were never greater than 23, and where I never
had more than 90 students in my 4 classes, I could only blame myself for those
low marks from so many.
Then, of course, when I invited other comments, some were
heartfelt: “Don’t call on us if we don’t want to talk right then,” or, “Don’t
favor students (no offence) but you kind of favored __ and __ and you always talked about what they did
great …. Do that with all the kids. I had some hurt feelings.” And some (I
think!) just humorous, such as admonishing this Language Arts teacher: “NEVER
start the morning (in homeroom) off with Language Arts. It put students in a
foul mood and then they don’t listen as much.” (FYI, that was merely #1 on this
student’s -thanks Brittney!- top 10
list of suggestions for me!)
Even more dangerous: inviting comments from high school students. I always promised not to read their
suggestions until the school year was over.
Thirty years later I am still miffed by the comment from a senior in a
Shakespeare class who suggested I take drugs to make me less uptight. I was
glad to see and talk with her (back
in the 1980’s surveys were done by hand, and by year’s end you knew everyone’s
handwriting) at her 10th high
school reunion, but even then it was clear—I had failed in some way with
several students…. Her words echoed what I often heard: Lighten up. A sense of humor never hurts.
**
I.
From “Lessons from
the Field: THE ROLE OF STUDENT SURVEYS IN TEACHER EVALUATION AND DEVELOPMENT”
-http://bellwethereducation.org/sites/default/files/Bellwether_StudentSurvey.pdf, May 2014.
States and districts are taking various approaches to measuring teacher
effectiveness…. While student surveys are not new, they typically have been
used to take the temperature of the whole school (for example, school climate
surveys) rather than focus on specific teachers….
But, because students are the direct consumers of instruction, with a
unique perspective on teacher impact, there
is a growing belief that students can provide valuable feedback on a teacher’s
performance in the classroom. Student perception surveys are increasingly seen
as a low-cost and reliable tool for gathering data and feedback on the quality
of teaching in individual classrooms. Well-designed student surveys ask
students about instructional practices that correlate with improved student
learning, such as student-teacher relationships, teacher management of the
classroom, rigor of lessons and student engagement, and teacher responsiveness
to student struggle. The feedback provides actionable information about
students’ classroom experiences….
… it was the Measures of Effective Teaching (MET) study, supported by
the Bill & Melinda Gates Foundation, which had the most direct impact on
the recent upswing in the adoption of student surveys. The three-year MET study
evaluated the use of student surveys in seven different school districts [including Denver Public Schools] … and
involved 3,000 teachers. The study concluded that … “students
know an effective classroom when they experience one.” In fact, surveys were
found to be broadly predictive of a teacher’s ability to increase student achievement
and were a more reliable measure than classroom observations. Importantly,
student surveys were also more stable year-to-year than student test scores,
meaning student survey results showed promise as an additional measure in
teacher evaluations to increase reliability and validity…. (Bold mine)
|
NCTQ’s
report included 15 “Lessons in Teacher Evaluation Policy.” #7 stated: “Surveys have emerged as an important
source of data and feedback on teacher performance. It is important for
states and districts not to underestimate what it takes to design a
high-quality instrument, and adopt validated instruments or get expert help
writing, testing and implementing surveys. … “
|
A recent report from the National Council on Teacher
Quality (NCTQ), a research and policy group, notes that many states are
incorporating surveys as an important source of data and feedback on teacher
performance. Specifically, 17 states now require or allow for use of student,
parent, or peer surveys in teacher evaluations. Twelve states require or allow
for student surveys in particular. (http://www.nctq.org/dmsView/State_of_the_States_2013_Using_Teacher_Evaluations_NCTQ_Report, p. 31. See
box. Required in 8 states: Alaska,
Connecticut, Georgia, Hawaii, Iowa, Massachusetts, New York, and Utah.)
[Four] other states [Colorado, Missouri, Mississippi, and New
Mexico] have stopped short of mandating student surveys but allow them to be used as one measure of
teacher effectiveness. For example,
in Colorado, the use of student
perception data is strongly encouraged in the state rules as a means to provide
teachers with feedback on their performance. [See box—happily, not required,
but we can….]
|
COLORADO:
“student survey results can be used….”
Britt
Wilkenfeld, Assistant Director of Research for Educator Effectiveness at the
Colorado Department of Education, wrote me: "Under current Colorado
statute and State Board of Education rules, student survey results can be
used for an educator’s evaluation as long as student academic growth still
accounts for at least 50% of the evaluation and all of the Teacher Quality
Standards are assessed."
|
Several large school districts have also either begun using—or plan to
use—student surveys about their experiences in individual classrooms as a means
to inform teacher evaluations and professional development.… Since the original study results were
published, four of the MET districts—Pittsburgh Public Schools, Denver Public Schools, Shelby County
Schools, and New York City Schools—have begun administering surveys
district-wide. (Bold mine) MORE ON DPS – Pages 6-7.
From
the report’s Conclusion – the “third
common measure”
The jury is still out, however, on whether student surveys will join classroom observations and student achievement data as a third
common measure in newly redesigned teacher evaluation systems, or if adoption
will remain limited to a small number of progressive districts and CMOs. Based on our research and interviews, we
believe that student surveys can be that third measure, because they add value that classroom observations
and student learning data cannot. … student survey data provide actionable
feedback that teachers and their supervisors can use to target specific areas
for improvement.
[This 2014 report pointed to the Colorado
Legacy Foundation’s piloting of a student perception survey “to determine
its validity and reliability.”[2]
The Foundation, now named the Colorado Education Initiative, has produced a
survey kit. Below is CEI’s material for grades 6-12. The version for use with
grades 3 – 5, can be found at http://www.coloradoedinitiative.org/toolkit/research/.]
**
II.
From “Colorado’s Student Perception Survey -
Grades 6-12,” Colorado Education Initiative http://www.coloradoedinitiative.org/wp-content/uploads/2014/09/SPS_Administration_survey-instrument-6-12-CEI.pdf, 2014.
Colorado’s Student Perception Survey[3]
is a 34 question instrument that
measures elements of student experience that have been demonstrated to
correlate most closely to a teacher’s ability to positively impact student
growth.[4] Students are asked to indicate how frequently
they experience each item with a response scale of always, most of the time,
some of the time, and never. … The survey items for grades 6
- 12 are listed below. (CEI’s website also includes “What the Research says
about Student Feedback for Teachers” - http://www.coloradoedinitiative.org/wp-content/uploads/2014/09/Planning_comms_Research-Overview-CEI.pdf.)
|
Thompson Valley is one of the
Colorado school districts using
CEI’s student perceptions surveys. (See
district web site- http://thompson.k12.co.us/Page/5721)
There we also see two examples of what teachers have said
in comparing fall and then spring results.
Cheerful notes on the good
progress I made this year, on my
success .… OK, but to make this work truly reflective—as in
self-critical—let’s be sure we don’t
simply pat ourselves on the back. If
the surveys are going to help us improve, we should trust the process, and
our students, in a way that allows us to see our shortcomings. Nobody’s
perfect! Reflecting on the not-so-good
news – that’s where can benefit so much.
|
Student Learning: How teachers use content and pedagogical
knowledge to help students learn, understand, and improve.
My teacher makes learning enjoyable.
What I learn in this class is useful to me in my real
life.
My teacher teaches things that are important to me.
My teacher knows the things that make me excited about
learning.
In this class, we learn a lot every day.
In this class, it is more important to understand the
lesson than to memorize the answers.
When the work is too hard, my teacher helps me keep
trying.
My teacher accepts nothing less than my best effort.
My teacher knows when we understand the lesson and
when we do not.
If I don't understand something, my teacher explains
it a different way.
My teacher explains difficult things clearly.
In this class, we have a say in what we learn and do.
My teacher talks to me about my work to help me
understand my mistakes.
My teacher writes notes on my work that help me
improve.
When we study a topic, my teacher makes connections to
other subjects or classes.
Student-Centered
Environment:
How teachers create an environment that responds to individual
students’
backgrounds, strengths, and interests.
My classroom is organized and I know where to find
what I need.
Students feel comfortable sharing their ideas in this
class.
My teacher respects my opinions and suggestions.
My teacher cares about me.
My teacher pays attention to what all students are
thinking and feeling.
My teacher respects my cultural background.
My teacher respects me as an individual.
Classroom Community: How teachers cultivate a
classroom learning community where student
differences
are valued.
|
From a
principal
“As a Head of School (I retired last
year), I introduced a formative teacher assessment process which included …
administering and reporting on a student questionnaire. Each question had a
Likert scale plus room for comment. The teacher was asked to analyse and
report on the data. While numbers of teachers expressed concern beforehand,
believing the responses would be immature, retributive or lacking
understanding, the reality was otherwise and teachers acknowledged that their
students filled out the surveys seriously and that they found the feedback
valuable. However, the process was formative.” - From
a comment on the blog, “Next Up in
Teacher Evaluations: Student Surveys,” Education
Week, 7/11/12.
|
My teacher would notice if something was bothering me.
Our
classroom materials (books, articles, videos, art,
music, posters, etc.) reflect my
cultural background.
In this class, I feel like I fit in.
I feel like an important part of this classroom
community.
My teacher knows what my life is like outside of
school.
My teacher knows what is important to me.
I ask for help when I need it.
I feel like I do a good job in this class.
Classroom Management: How teachers foster a respectful and
predictable learning environment.
Our class stays busy and does not waste time.
Students in this class treat the teacher with respect.
The students behave the way my teacher wants them to.
Students in this class respect each other’s
differences.
III.
From “Student Perception Surveys and Teacher
Assessments,” Hanover Research. https://dese.mo.gov/sites/default/files/Hanover-Research-Student-Surveys.pdf, Prepared
for XYZ, February 2013.
MY STUDENT SURVEY
My Student Survey was developed by Vanderbilt
University researcher Ryan Balch, and is based on research-based teaching
practices. The survey was validated in a large-scale pilot as part of Georgia’s
Race to the Top initiative in spring 2011. The pilot study administered the
survey to over 12,000 students in grades 6-12 at seven Georgia school
districts. The results of the pilot study suggest that My Student Survey is a
reliable evaluation of teacher effectiveness. The research revealed a
relationship between teacher ratings and academic student engagement and
self-efficacy. (Bold mine).
The survey questions are organized into
categories loosely based on two classroom observation frameworks…. In order to provide the most meaningful
feedback to teachers, Balch then grouped questions into six constructs that
represent the role of a teacher:
Presenter Manager
Counselor Coach Motivator Content Expert
[Here are sample questions for each of the constructs.] The survey includes 55 questions that students
rate using a five point frequency-based Likert scale, ranging from “Never” to
“Every time.”
My Student Survey Sample Questions
|
CONSTRUCT
|
SAMPLE QUESTION
|
|
PRESENTER
|
When explaining new skills or ideas in
class, my teacher tells us about common mistakes that students might make
|
|
At the end of each lesson, the teacher
reviews what we have just learned
|
|
|
MANAGER
|
My teacher corrects students when they do
not follow the rules of the class
|
|
We are learning or working during the
entire class period
|
|
|
COUNSELOR
|
My teacher shows respect for all students
|
|
My teacher notices when I am not
participating in class
|
|
|
COACH
|
My teacher gives us guidelines for
assignments so we know how we will be graded (grading rules, charts, rubrics,
etc.)
|
|
I have to work hard to do well in this
class
|
|
|
MOTIVATOR
|
My teacher has us apply what we are
learning to real-life situations
|
|
My teacher encourages me to share my ideas
or opinions about what we are learning in class
|
|
|
CONTENT
EXPERT
|
My teacher is able to answer students’
questions about the subject
|
|
After asking us questions, my teacher lets
us think for a few seconds before we have to answer
|
TEACHER EVALUATION
… Researchers agree that
student surveys should not be the only factor in teachers’ evaluations; however, their reliability suggests their use is
appropriate when combined with other measures such as classroom observations
and value-added student achievement gains. (Bold mine)
Although the use of
surveys is limited to a small number of districts throughout the country, the
trend is moving in favor of more comprehensive evaluations and an increasing
number of states and districts are using student surveys. (p. 10)
DENVER PUBLIC SCHOOLS
Denver
Public Schools took part in the Bill & Melinda Gates Foundation’s MET
Project pilot study beginning in 2009. Although Denver administers the survey
district-wide to over 60,000 students annually, the surveys are still in pilot
form and are used for professional development only, not summative teacher
evaluations. The pilot was implemented within Denver’s Framework for Effective
Teaching…. A major component of this initiative is the
Leading Effective Academic Practice (LEAP) pilot,[5]
which incorporates five elements into
its performance assessment system: student outcomes, principal observation, peer
observation, professionalism, and student perception surveys. Student
outcomes remain the primary source of evaluation, but student perception
surveys appear to count for 5 percent of the overall performance assessment….
…
The pilot program website explains that “LEAP provides teachers with additional
feedback and support so they can continue to learn and grow professionally.”
Its system of multiple measures is
designed
to provide teachers with a “comprehensive body of evidence” so they can
identify
strengths,
weaknesses, and progress. The student perception surveys play a key role in
this system, and the district claims
that its teachers have found great value in the survey data and modified their
practice as a result. (Pages
19-20. Bold mine)
IV.
From “Gathering Feedback from Students” at Vanderbilt
University’s Center for Teaching, - https://cft.vanderbilt.edu/guides-sub-pages/student-feedback/, downloaded
May 2016.
This web site includes many examples of both In-class
Feedback Forms and Online Surveys. Here’s one:
Course:
____________________________ Instructor
Name: ____________________________
1
= Never; 7 = Frequently
|
1
|
Indicates
where the class is going
|
1
|
2
|
3
|
4
|
5
|
6
|
7
|
|
2
|
Explains
material clearly
|
1
|
2
|
3
|
4
|
5
|
6
|
7
|
|
3
|
Indicates
important points to remember
|
1
|
2
|
3
|
4
|
5
|
6
|
7
|
|
4
|
Shows
genuine interest in students
|
1
|
2
|
3
|
4
|
5
|
6
|
7
|
|
5
|
Effectively
directs and stimulates discussion
|
1
|
2
|
3
|
4
|
5
|
6
|
7
|
|
6
|
Provides
helpful comments on papers and exams
|
1
|
2
|
3
|
4
|
5
|
6
|
7
|
|
7
|
Is
tolerant of different opinions expressed in class
|
1
|
2
|
3
|
4
|
5
|
6
|
7
|
|
8
|
Is
available outside of class
|
1
|
2
|
3
|
4
|
5
|
6
|
7
|
|
9
|
Explains
thinking behind statements
|
1
|
2
|
3
|
4
|
5
|
6
|
7
|
|
10
|
Effectively
encourages students to ask questions and give answers
|
1
|
2
|
3
|
4
|
5
|
6
|
7
|
|
11
|
Adjusts
pace of class to the students' level of understanding
|
1
|
2
|
3
|
4
|
5
|
6
|
7
|
|
12
|
Seems
well-prepared
|
1
|
2
|
3
|
4
|
5
|
6
|
7
|
|
13
|
Stimulates
interest in material
|
1
|
2
|
3
|
4
|
5
|
6
|
7
|
|
14
|
Treats
students with respect
|
1
|
2
|
3
|
4
|
5
|
6
|
7
|
|
15
|
Is
effective, overall, in helping me learn
|
1
|
2
|
3
|
4
|
5
|
6
|
7
|
What do you think is this instructor's greatest strength?
What suggestions would you give to improve this instructor's teaching?
V. From “Companies Honing Tools to Survey Students About Teachers.” Education Week,
[The
article provides a helpful overview of what is taking place on student surveys.
It lists five different survey providers. Two quotes from the article:]
Steve Cantrell, a lead senior program officer
for the Gates foundation, said he recognizes teachers' wariness of using surveys
for evaluations. But he noted that student surveys were the "most reliable
measure" of teachers' effectiveness included in the Gates study. Those
results were more consistent, for instance, than classroom observations of
teachers or their performance as judged by student test scores, he said. "We're
not asking students to judge the quality of the teaching. We're asking them to judge the instructional
environment. ...This is information that teachers can't get from anyone else
but the students."
Nonprofit organizations, like Battelle for Kids,
have also become interested in the information that
can be gleaned from student surveys. In partnership with the Gallup polling
organization, the Columbus, Ohio-based Battelle offers an online survey, which
can be taken by students on computers or mobile devices, designed to measure students' sense
of hope, engagement, and belonging—as well
as teachers' classroom management. Battelle and Gallup are convinced that
students' belief in their learning environment is crucial to their academic and
long-term success. (See box for Battelle’s web site; also see http://www.thestudentexperience.org/)
|
Battelle
for Kids
We
developed The Student Experience® Survey in
collaboration with Gallup® to capture classroom-level
feedback from students to help teachers create a learning experience that
supports student success now and in the future. The survey captures feedback
around four key themes:
Designed for students in grades
4–12, the research-based 24-item survey can be completed online in less than
15 minutes, protecting instructional time while providing timely feedback to
support improvement. Educators can customize the survey to get a snapshot of
the student learning experience by classroom, course, class period, or
grade.
|
"Students won't learn as much if they don't
feel they belong in the classroom," said Tim Hodges, the director of
research for Gallup Education. "It's a false choice [when] people say it's
[only] the soft stuff, or the academic measures, that matter" to students'
success.
Addendum A
Teacher
evaluation (a form I created and used – borrowing from many others I had seen)
Please circle one
number that represents your view.
1 - strongly disagree 2 –
disagree 3 – agree
4 – strongly agree
1.
Overall, the teacher
presented material in a clear manner. 1 2 3 4
2.
The teacher presented and discussed the literature in a clear
manner. 1 2
3 4
3.
The teacher presented and discussed writing skills in a clear manner. 1 2
3 4
4.
The teacher was supportive of my efforts to improve as a reader and
writer. 1 2 3 4
5.
The teacher helped motivate me to do better. 1
2 3 4
6.
The classroom environment was friendly and positive. 1 2 3 4
7.
There was sufficient variety in the kind of classroom activities that
took place. 1 2
3 4
8.
The teacher made an effort to know me personally and to teach me as an
individual. 1 2 3
4
9. The teacher did not lecture, or give
uninterrupted talks, too often. 1 2 3
4
10. The teacher
demonstrated effective classroom management skills to help keep us on task.
1 2 3 4
11.
The teacher’s approach to discipline was fair and effective. 1 2
3 4
12.
The teacher listened to my questions and those of my classmates with
respect, and the teacher did a good job of making us feel we could participate. 1 2 3 4
13.
Overall, I felt that I was treated with respect by the teacher. 1 2 3 4
14.
In discussions, the teacher treated my views and the views of my
classmates with respect, and we felt we could speak honestly. 1
2 3 4
15.
I have felt comfortable contributing to discussion in this class. 1
2 3 4
16.
When the teacher spoke about my work in class, the teacher did not
embarrass me. 1 2
3 4
17.
The teacher gave out grades that seemed fair. 1 2 3
4
18. The teacher’s
grading system was consistent with most other teachers in the middle school. 1
2 3 4
If you indicate 1 or 2, circle one of
the two below, indicating that this teacher’s grading system was harder or easier.
[1] See Addendum A
for the two-page survey I used 10 years ago, for middle school students.
[2] See CEI’s
141-page STUDENT PERCEPTION SURVEY
TECHNICAL REPORT, http://www.coloradoedinitiative.org/wp-content/uploads/2014/09/technical-report-CEI.pdf. Colorado Education Initiative. (2013). Denver,
CO: Author.
[4] A number of
items on the Colorado SPS were adapted from items made available for
non-commercial use through the Measures of Effective Teaching (MET) Project,
funded by the Bill & Melinda Gates Foundation.
[5] This was written in 2013. Today, LEAP is well past the “pilot” stage.
No comments:
Post a Comment