Tuesday, September 3, 2019

AV #199 - When excessive credit for “growth” = “the soft bigotry of low expectations”



As we revise our accountability framework, we need to raise the bar

Close to 85% of our K-12 schools “earn” the top two SPF ratings. How is this possible?

Colorado’s accountability framework places a significant emphasis on growth scores. This seems fair, especially in light of what we know to be the correlation between achievement and a school’s zip code. No one wants to be seen as judging a low-income, low-performing school harshly; we must respect improvement, no matter where the starting line. I appreciate why growth matters.

But what if the increase in the percentage of students in one elementary school “meeting or exceeding expectations” in mathematics is tragically small (from 4.2% to 6.4%)—but somehow our algorithms leads this school to “earn” a growth rating of 66% points? What if our very approach—“Student Growth Percentiles are determined by how much students have progressed compared to their ‘academic peers’[i]—sets the bar too low? And what if the “good news,” on one growth measure, appears to explain why this one school is granted a (preliminary) rating, in 2019, of IMPROVEMENT? And—equally hard to credit—why its SPF rating jumps from 42.9 % points earned in 2018 to 51 % points earned in 2019!

Or to state the question another way, what if our well-meaning desire to be fair (or nice, or generous, or sympathetic) actually looks a lot like “the soft bigotry of low expectations”?

“What we were looking for is some alignment,” said Angelika Schroeder, chair of the State Board of Education. “When you have 73% of schools that are at performance and the (state test) results are what they are, there’s a big disconnect.”[ii]
That familiar phrase from Michael Gerson stings. We insist it cannot be not true of us. But I have been trying to understand why a good number of extremely low-performing schools are, for now anyway, on the Preliminary School Ratings for 2019, on IMPROVEMENT. The culprit, as I will call it, seems to be the excessive credit schools are given for the most miniscule growth. Aren’t we telling these schools and their students how little we expect? The slightest bump up—and we say: good enough! The district results look better (one less school on PRIORITY IMPROVEMENT or TURNAROUND). The superintendent and the district look smarter (such a good decision, the Aurora school board might say, to put this school on innovation status back in March 2016[iii]). All is well.  

One begins to wonder if achievement scores even matter.

I fear getting into the weeds in this close look at two schools—perhaps only of interest to those of you implementing HB 18-1355[iv]  or on the Technical Advisory Panel on Longitudinal Growth.  (Some good ideas are on the table.[v]) But I hope my larger point concerns many. In trying to be “fair,” I believe we present a false picture of how our K-12 schools are doing.

The headline in Chalkbeat Colorado last December read: “State ratings identify 163 Colorado schools in need of improvement.”[vi] In August, preliminary results for 2019 showed less than 131 schools with the lowest two ratings.[vii] Really? When we know the number of schools struggling to help even 25% of their students meet our standards is much greater? I think we have good reason to tell policymakers over 200 schools should not be rated on IMPROVEMENT or PERFORMANCE.  Why not alert Gov. Jared Polis and legislators to this reality? They might realize more resources are needed for the Colorado Department of Education’s School Quality and Support Division and for our lowest-performing schools. But first we have to name them as such. All of them.

Here are some details on two schools, both rated on IMPROVEMENT, to make my point. (They are hardly alone. See 15 more.[viii]) If the facts below are right, are these not schools we must identify as in trouble – and in great need of support?

Another View has studied Paris Elementary School since 2015[ix]. Chalkbeat Colorado’s snapshot for 2019 reads[x]:

Literacy - % Met or Exceeded Expectations          6.7
Math - % Met or Exceeded Expectations               6.4
Literacy Growth               38.5
Math Growth                    66

Please stare at those achievement results. A school that has earned our second highest SPF rating? Look at the ELA and MATH scores over the past two years. How does that 66 in Math Growth look now?


2018
2019
2018
2019

Percent of students
meeting or exceeding expectations
Number of students
meeting or exceeding expectations
ELA
9.9%
6.7%
19
11 – out of 164 tested
MATH
4.2%
6.4%
8
11 – out of 171 tested


   “…many high-poverty schools with impressive growth scores still have many students who can’t read or do math at grade-level, leaving parents and education activists fearful for their long-term prospects.
   “‘Outcomes matter to families,’ Nicholas Martinez, co-founder of the advocacy group Transform Education Now, said bluntly at a recent state board meeting.” “Changes pending for Colorado school rating system”[xi]
So 11 out of 171 Paris Elementary students who took the CMAS Math in 2019 scored high enough to meet or exceed expectations. All of 3 more students than in 2018. And yet the school’s Math Growth score (which only includes progress from grade 3 to grade 4, and from grade 4 to 5) is well above average – 66. (Median Growth Percentile (MGP) explained: see Endnote.[xii])

Focus on that 6.4%. And what about the other 93.6%? On the 2019 CMAS, few were even Approaching Expectations!

ELA: 81.3% of students scored in the lowest 2 categories: Did Not Meet Expectations or Partially Met Expectations. (That is a higher percentage than in 2018, when 73.4% scored in the lowest two categories.)
MATH: 76% scored in the lowest 2 categories: Did Not Meet Expectations or Partially Met Expectations. (Not much different from 2018, when 78.4% of the students scored in the lowest two categories.)

**

We see a similar story over in Denver Public Schools with Cheltenham Elementary School. Also rated on IMPROVEMENT. Again, here is the overview on Cheltenham Elementary in 2019 from Chalkbeat Colorado[xiii]:

Literacy - % Met or Exceeded Expectations          20.4
Math - % Met or Exceeded Expectations               16
Literacy Growth               53
Math Growth                    62

CDE’s more detailed figures[xiv] invites us to wonder what leads to such an “impressive” growth score.


2018
2019
2018
2019

Percent of students
meeting or exceeding expectations
Number of students
meeting or exceeding expectations
ELA
19.3%
20.4%
33
30 – out of 147 tested
MATH
12.7%
16%
24
26 – out of 163 tested

An even closer look—for each of the 3 grades tested—shows that the percentage of students meeting expectations declined in grades 3 and 4, but 5th grade scores were much better—and thus, apparently, the MGP scores exceeded 50. 

Cheltenham Elementary – CMAS 2018 and 2019 - % meeting or exceeding expectations

Subject
Grades
2018
2019
Change
3
17.4
15.2
-2.2

4
23.4
10.4
-13.0
  
5
16.4
30.3
+13.9
             Overall

19.3
20.4
+1.1
MATH
3
14.5
11.9
-2.6

4
10.6
7.3
-3.3

5
13.1
25.8
+10.6
            Overall   

12.7
16
+3.3
 









Congratulations to the fifth-grade teachers and students. But how much should we make of the MGP scores, in light of the school’s achievement scores falling in two of the three grades assessed? The state’s algorithms must give inordinate importance to growth—in one grade. For according to the 2019 preliminary ratings, Cheltenham “earns” the state’s highest rating: PERFORMANCE, with 59.3% points, 17 points higher than the year before!

   “State officials say that a school’s growth number — officially called the median growth percentile — ideally should be higher than the state median, which is about 50. But they say it’s also important to consider a school’s overall achievement level when examining growth data.”[xvi]
   Yes, and this is exactly what I am doing. Looking at achievement.







**

Compare and contrast: when we tell a school NOT GOOD ENOUGH, perhaps a greater urgency to improve

Finally, a related comment on the “unfairness” of placing schools on PRIORITY IMPROVEMENT or TURNAROUND, when it is nicer of us to create formulas that lift schools “on the bubble” into the (higher) category of IMPROVEMENT. In last January’s AV #188[xvi], my look at Paris Elementary and the 2018 SPF ratings, I pointed to a number of schools (like Paris) that earned less than 43 percentage points, but that were rated on IMPROVEMENT. Nine months later, I pose this question: Did the state’s (too) generous rating prove helpful?

School
District
2018
2019
West Elementary
Colorado Springs 11
Improvement - 42.6
Priority Improvement - 37.8
Farrell Howell ECE-8      
Denver Public Schools
Improvement – 42.8
Turnaround – 29.9
Englewood Middle School
Englewood
Improvement – 42.8
Turnaround - 33

In contrast, what happened in the three Denver schools, also mentioned in AV #188, where the state SPF deferred to the tougher district rating DPS has often maintained? Perhaps the PI label got their attention.

Improved School Performance Rating, after being put “on the clock” - from 2018 to 2019
School
District
2018
2019
Monarch Montessori
DPS
Priority Improvement - 42
Improvement – 52.3
Stedman Elementary
DPS
Priority Improvement - 42.4
Performance – 53.1
STRIVE-GVR
DPS
Priority Improvement - 43
Performance - 53

Many adults are pleased when a chronically low-performing school is able to “escape” the two lowest ratings. I would ask if we are doing the students and their families a favor. Aren’t we willing to tell it like it is?

**

Endnotes



[v] “Colorado isn’t doing away with growth measures, but all the proposals would add a new metric that measures how long it would take a student growing at a particular rate to move up one level. State education officials want to see growth rates that would allow a student who partially meets grade-level expectations in third grade to be at least approaching expectations two years later, in fifth grade, and for the fifth-grader who is approaching expectations to have met them by seventh grade.
“This catch-up measurement, a version of what’s called ‘growth to standard,’ would count for 10% of elementary and middle school ratings, with the previous growth score counting for 55% and achievement for 35%. The changes to high school ratings are still a year or more away.”
[viii]  Each of the following schools received a preliminary SPF rating for 2019 of IMPROVEMENT:

District
CMAS -ELA – 2018
CMAS - MATH - 2019
Growth of 50 or better
Adams 14
Adams Middle
21.3
8.2


Hanson Elementary
20.8
9.8
Literacy Growth-62

Kearney Elementary
18.6
11.8

Aurora Public Schools
Crawford Elem.
10.4
13.1
Literary Growth-52*

Kenton Elementary
18.6
9.8
Literacy Growth - 50

Lansing Elementary
17.0
9.5
Literacy Growth – 52
Math Growth - 58

Peoria Elementary
17.6
16.2


Virginia Court Elem.
13.4
12.2
Math Growth - 51
Charter School Institute
Montessori Del Mundo Charter
17.3
8.6
Math Growth - 50
Denver Public Schools
Ashley Elementary
23.7
13.6
Literacy Growth – 50.5

Colfax Elementary
20.9
22.0


Johnson Elementary
21.4
12.6

Greeley 6
Franklin Middle
21.2
12.1
Literacy Growth - 51
Jefferson County
Fitzmorris Elem.
24.4
13.0
Literacy Growth - 50
Pueblo 60
Beulah Heights Elem.
18.6
13.2
Math Growth - 58
[ix]AV #126 (March 2015), AV # 177 (March 2018), and AV # 188 (Jan 2019). Final paragraph from #188 reads: “The SPF, like any human creation, is fallible and sometimes gets it terribly wrong. If the goal has been to help provide better ‘support and intervention’ to our lowest-performing schools and ‘enhance … oversight of improvement efforts,’ the SPF needs to make it clear that schools like Paris Elementary are not doing OK. They do not need the stamp of on Improvement. They need help.”

[x] "Find your 2019 Colorado CMAS scores — and compare schools,” https://www.chalkbeat.org/posts/co/2019/08/15/find-your-2019-colorado-cmas-scores-and-compare-schools/.

[xvii] AV #188 – “School accountability in Colorado - No need to abandon the School Performance Framework, but it is far from perfect. Case in point: Paris Elementary and student achievement” (January 23, 2019).  https://anotherviewphj.blogspot.com/2019/01/av188-school-accountability-in-colorado.html.


No comments:

Post a Comment