Wednesday, September 6, 2017

AV#167 - Even LESS evidence now to grant innovation status to low-performing schools


                                                   
Submitted to: Governor John Hickenlooper
House of Representatives Education Committee
Senate Education Committee
2) “After years of stagnation, Colorado’s innovation schools see breakthrough in improvement, data show,” by Nic Garcia, Chalkbeat Colorado (Aug. 2, 2017).[ii]
I wrote AV#159 last spring, “When on the clock – Innovation status to the rescue – on what basis?” to urge state and district school board members to say—Hold on! Look at the results!   Does the Colorado Department of Education’s 2017 report on Innovation Schools, or an analysis of it last month by Chalkbeat Colorado, change my mind? Not at all.  Both reports, I believe, make a small but critical error.

I am thrilled if AV#159 played any role in dissuading Adams 14 from following in the footsteps of Aurora and Pueblo – two districts that had decided by 2016 that innovation status would be a way to foster improvement in some of their lowest performing schools. Based on almost no evidence, as I showed - http://anotherviewphj.blogspot.com/.  

On the other hand, in April the state board approved of Greeley’s plan to provide “new autonomy for two struggling Greeley middle schools as part of its work this spring to turn around chronically low performing schools.”[iii]  And in May the board allowed Aurora Public School’s plan for Aurora Central High–giving “the school more autonomy”[iv]—to go forward.  So the argument still needs to be made.                                                                   

The flaw I see in CDE’s report pertains to efforts in (fortunately, just a handful of) districts to employ innovation status as a means to dramatically improve their struggling schools.  More freedom for schools doing fairly well – I’m all for it.  But to think that the schools unable to bring about significant academic improvement will be better off on innovation status—no; I still see no evidence to support this. And Chalkbeat’s summary—and the article’s headline—seems misleading.  A “breakthrough”?

One-size reform strategies (innovation schools, AP classes) forget this truth: context matters

Another View has consistently pushed for greater school autonomy. But context matters. As with the points I have made about Advanced Placement courses, we cannot assume that what proves beneficial to our more successful schools is well-suited to those chronically under-performing. We have made so many errors in our turnaround work; let’s not make this one too.  What have we learned about effective efforts?  Such schools need greater support, even intervention—not letting go.  Hands off? No, these schools need all-hands-on-deck.  The state, district, outside partners—anyone who can help.

CDE’s 50-page report itself offers practically no judgments. It lists 86 schools, the year they gained innovation status, and how their School Performance Ratings have changed, up or down, over time.  It helps us see the big picture, revealing what has taken place in the schools in Colorado, mostly in Denver Public Schools, that have been on innovation status for five or six years.

It may seem petty of me to “take issue” with a solid report when practically all I object is to is one concluding paragraph in its 14-page narrative—before the 36-page Appendix:

Colorado has seen rapid growth in the number of innovation schools and innovation zones across the state. The number of innovation schools has increased 39 percent since this report was published last year.…  More recently, the Department has seen significant interest from districts and schools wanting to learn about the Innovation Act as they consider innovation as a means of turnaround for schools approaching the end of the five-year accountability clock (as per SB 09-163). (Bold mine)
And yet the report provides no proof that greater autonomy has been a way to improve low-performing schools.  I believe it is a mistake not to say that to policymakers.  Just as it would have been a mistake to let Adams 14 stumble down this dead-end road.  My own Addenda will show that the far majority of innovation schools that have moved forward well were rated on Improvement and Performance when they received innovation status—not schools on Turnaround and Priority Improvement.  (And really, does this surprise anyone?  Isn’t this just common sense?)

**
To substantiate my argument in AV#159, I quoted from Chalkbeat’s May 2016 article—"Freeing failing schools from bureaucracy hasn’t worked as hoped. So why is Colorado still doing it?”  Last month, Chalkbeat began its analysis of CDE’s Innovation study by referring to that 2016 article:
A year ago, as the State Board of Education began to consider fixes for Colorado’s lowest-performing schools, there was scant evidence that giving schools freedom from some state laws and local policies would significantly boost learning.

But Chalkbeat has now come up with a more positive outlook. In August it reported:
Ten schools with innovation status improved enough between 2014 and 2016 to avoid state-ordered improvements. The near quadrupling of schools that improved … could bolster backers of giving schools greater decision-making authority….
A paragraph that follows, however, undermines this “good news”:
Some Colorado schools with innovation status continue to struggle. Seven innovation schools continue to rank among the lowest-performing in the state after multiple years of increased autonomy. And four innovation schools that had one of the state’s highest ratings in 2014 dropped to one of the lowest in 2016.(ii)

My Addenda presents key points from CDE’s report. It shows that, of our 86 Innovation Schools, every one of them not in Aurora, DPS, and Pueblo were already seen as relatively healthy schools when they sought and gained Innovation status (Addendum A-page 3).  All but two were rated on Performance (highest rating) that year; the other two were rated on Improvement (second highest rating).  We cannot look to these other 10 districts and 26 schools for success or failure regarding the use of Innovation Status to help bring about improvement in low-performing schools

What about the ten schools Chalkbeat saw emerge from being “on the clock”? (See Addendum B-page 3). As Chalkbeat noted, most of those–eight, to be exact–are in Denver (NOTE: 8 out of 47 innovation schools in DPS). Addendum C-“A far more muddled picture”-page 4, looks at 21 other DPS schools.

The other two schools on Chalkbeat’s list are in Pueblo 60.  AV#159 questioned the so-called improvement at Pueblo’s three middle schools on innovation, based on 2016 PARCC scores.  Here we go again: look at 2017 PARRC results all three (Addendum D-Pueblo-page 6).  Too grim for anyone to call this a success.  As for Aurora Public Schools?  Too soon for any significant conclusions, but a quick look at PARCC results at three schools on innovation offers more red flags—and the same warning: you are headed in the wrong direction! (Addendum E-Aurora-page 9).  

I am sure we all care about addressing the persistent challenges of our lowest-performing schools.  But innovation status is no answer. We know better. To be more precise - by now, we should know better.  


Addendum A

In most districts (10 of 13), schools were not low-performing when granted Innovation status

All but two schools listed below were rated on Performance (highest rating) the year they received Innovation status.  The other two were rated on Improvement (the second highest rating).


# of Innovation schools
Granted Innovation status when rated on Performance
Granted Innovation status when rated on Improvement
Burlington
3
2
1
Delta
1
ALL

Greeley-Evans
1
ALL

Falcon
11
ALL

Holyoke
3
ALL

Kit Carson
2
ALL

Mancos
3
2
1
Montrose County
1
ALL

Widefield
1
ALL

TOTAL
26
24
2
Westminster
1
Granted Innovation Status before it opened


Addendum B

Chalkbeat’s “10 schools” now rated higher - on Improvement or Performance

Only three districts have tried innovation status as a strategy for low-performing schools. Here are the 10 schools from two* of those districts (DPS and Pueblo 60) referred to in Chalkbeat’s analysis. Half of them received innovation status when they were rated on Turnaround or Priority Improvement. All 10 are now on Improvement or Performance.

                          From CDE’s 2017 Innovation Schools Annual Report: 

School District/ School
Year it received Innovation Status & Rating - or
Opened with IS

After Innovation Status

2016 Rating
Total % pts earned -2016
-
AECs have AEC pts earned**
Denver



Centennial
2012   
TURNAROUND
2013 & 2014
 TURNAROUND

Improvement

59.5
Collegiate Prep
  Opened with IS
2012-Performance; 2013 Improvement
2014 – Rating dropped to Priority Improvement

Performance

53.9
Compassion Road
 Opened with IS
2014 – TURNAROUND
AEC: Improvement
Plan

50
DCIS at Ford
Opened with IS
2013 & 2014 - Priority Improvement
Improvement

51.8
Denver Center for 21st Century Learning at Wyman
Opened with IS
2012-2014:
AEC: TURNAROUND
AEC: Improvement Plan

40.46
Denver Montessori
Opened with IS
2014 - TURNAROUND
Performance
66.2
Summit
2011   
AEC: TURNAROUND
2012-2014 –
AEC: TURNAROUND
AEC: Improvement Plan
       
        50.77
Trevista K-8
2012
Priority Improvement
2013 - Declined to TURNAROUND;
2014 on Priority Improvement

Performance

57.3
Pueblo




Pueblo Arts
2013 - Improvement
2014 -declined to TURNAROUND
Improvement Plan
43.1*
Roncalli
2013   TURNAROUND
2014 -  TURNAROUND
Improvement Plan: Low Participation
46.1*
*Aurora Public Schools is the third district – see Addendum E.
**AEC – Alternative Education Campus

As I review those ratings, I would say well done to just three schools -  Centennial, Denver Montessori, and Trevista.  To Collegiate, good to see you get back on track after a poor performance in 2014.  To the others … do we really see much to celebrate?  [Preliminary 2017 SPF ratings are lower at three of these four schools:  Centennial, Denver Montessori, and Collegiate.[vi]]

CDE’s Innovation Schools Annual Report shows one other DPS school, Goldrick Elementary, climbed out of an SPF rating of Turnaround or Priority Improvement in 2014, up to Improvement or Performance in 2016.  But it was not on innovation status until 2016.


21 other Denver Public Schools on Innovation

Ten schools (or is it really only three?) hardly tells us a lot. Let’s take a broader look.

Not one of these 21 DPS schools went from one of the two lower ratings - Turnaround/ Priority Improvement - to Improvement/Performance between 2014 and 2016

From CDE’s 2017 Innovation Schools Annual Report
(with my changes based on finding errors in the report—i.e. there was no SPF for 2015)


Year granted Innovation designation
SPF rating after being on Innovation
SPF Rating 2016
Ashley Elementary
Innovation status in 2013 when on Priority Improvement
Rating lifted to Improvement in 2014
Rating dropped to
Priority Improvement
Denver Green School
Opened on Innovation status 
On Priority Improvement in 2011, then on Performance next three years

Performance
Excel Academy
Opened on Innovation status
First year on
AEC: Turnaround Plan

AEC: Turnaround Plan
Godsman Elementary
Innovation status in 2011 when on Priority Improvement
On Performance next three years, 2012-14
Rating dropped to Improvement
Grant Beacon
Innovation status in 2012 when on Improvement
Has been on Performance all three years since:
2013, 2014, 2016
Green Valley Elementary
Innovation status in 2011 when on Improvement
Spent next four years on Performance
Rating dropped to Improvement
High Tech Early College
Opened on Innovation status
Spent next two years on Performance
Last two years rating dropped to Improvement
Isabella Bird Community School
Opened in 2013 – granted Innovation status
2014
Performance
Rating dropped to Improvement
Joe Shoemaker
Opened in 2015 –
granted Innovation status

DDP - SPF rating */
Turnaround
Legacy Options
Opened in 2015 – granted Innovation status

DDP - SPF rating */
Turnaround
McGlone
Innovation status in 2011 when on Improvement

Has been on Performance all four years since – 2012-2016

Manual High School
Granted Innovation status in 2009
2010-12 on Improvement
2013-14 rating dropped to Turnaround

DDP-SPF rating */
Priority Improvement
Noel Community Arts School
Opened on Innovation status
On Improvement first two years – 2012 and 2013;
Rating dropped to Turnaround in 2014
Priority Improvement
Oakland Elementary
Opened in 2013 – granted Innovation status
2014
Turnaround

Turnaround
Place Bridge
Granted Innovation status in 2015 after 3 yrs on Performance 2012-14


Performance
Schmitt Elementary
Granted Innovation status in 2016
2014
Turnaround

Priority Improvement
Swigert International School
Opened on Innovation status
Has been on Performance all four years – 2012-2016

Valdez Elementary
Granted Innovation status in 2010 when on Improvement
2011-13 on Performance;
2014 rated dropped to Improvement
Rating back up to Performance
Valverde Elementary
Granted Innovation status in 2016

Turnaround
West Early College
Opened on Innovation status
Has been on Turnaround all three years – 2013,2014, 2016

Whittier K-8 School
Granted Innovation status in 2010
when on Improvement
Next two years on Performance – 2011, 2012;
Rating dropped to Improvement 2013, 2014
Rating back up to Performance
*“DDP - SPF rating decreased due to participation on state assessments” (CDE’s Innovation Report).


CDE gives a shout out to McGlone and Grant Beacon – because…?

CDE’s report goes out of its way to be neutral on whether Innovation Status is good or bad.  I find just one sentence in CDE’s that reflects a positive comment on the academic learning at two of the 86 schools.  They are a strange choice.

Grant Beacon Middle School and McGlone Elementary School, both from Denver Public Schools, are two examples of schools who pursued innovation as a means of being able to better meet the needs of their students. Both schools have made significant improvements over the last 5-6 years, as reflected by their School Performance Framework, since converting to innovation schools.[vii]

But did they really make “significant improvement … since converting to innovation schools”?  Grant Beacon became an Innovation school in 2012 when it was rated on Improvement (2012) – the state’s second highest rating. Ever since then, for three straight years, it has been on Performance (highest rating).  McGlone was also rated on Improvement when it became an Innovation School (2011), and it has been on Performance ever since - for four straight years – 2012-2016.  [Preliminary 2017 ratings show McGlone’s rating has dropped to Improvement.]

Again, I see two schools that were doing fairly well when they became Innovation schools.  Perhaps the waivers granted through the innovation status helped them show even stronger results.  But they cannot be used as examples of what will help schools on Turnaround and Priority Improvement.


Addendum D – Pueblo

Three turnaround schools on the right track?

This passage from AV#159 (March 2017) indicates my doubts about the “improvement” in Pueblo’s three Innovation Schools, looking at the 2016 PARCC scores.  Before the Colorado State Board of Education approved of Pueblo’s request to set up an innovation zone, “one could look at the PARCC results at the three schools (already on Innovation) and compare 2015 and 2016 ELA and Math scores”: 
·      8th grade scores in 2016 were better than in 2015, highlighted by a nice jump at Pueblo Academy ….  But that was the only case where over 15% of the 8th graders at any of three schools were proficient, in either English or math. 
·      English scores for 6th and 7th grades declined at Pueblo Academy of the Arts and at Risley International from 2015.
·      Math scores dropped badly at these two schools for 6th grade and were just slightly better for grade 7; still, less than 10% of the students were proficient: 9.1% at Pueblo Academy of the Arts; 8.8% at Risley. At Roncalli, even though all three grades “improved” their math results, less than 10% were proficient for each grade. (http://www.chalkbeat.org/posts/co/2016/09/01/find-your-schools-2016-parcc-english-and-math-results/)

“Isn’t it true,” I wrote last March, “that press reports and district claims overstated the ‘improvement’ and ‘success’ at the first three schools?”  I quoted Interim Superintendent Charlotte Macaluso’s positive spin on the 2016 results for these three schools, but I was not convinced:

OK, some good news at Roncalli.   And I realize that in January 2017, when CDE released its 2016 School Performance Framework, it lifted the ranking for both Roncalli STEM and Pueblo Academy of the Arts to Improvement—a big leap from being on Turnaround in 2014. 
However, note that in previous years the state’s School Performance Framework usually put schools earning below 47% points on Priority Improvement.  The new guideline lowers that bar to below 42% points. Roncalli earned 46.1% points in 2016 and Pueblo Academy, 43.3% points; as a result, both gained a much higher ranking.  [See] The PARCC scores above … and the schools are now on Improvement? Really?

**

For its dubiously titled article in August, “Colorado’s innovation schools see breakthrough in improvement,” Chalkbeat spoke with Roncalli’s principal: (bold mine)
Marci Imes, principal of Roncalli STEM Academy, a Pueblo middle school that was one of the innovation schools that improved enough to get off the watch list, said time, focus and years of hard work were necessary to raise her school’s quality rating.
Again I cannot see why Roncalli came “off the watch list” in 2016.  Its overall declining PARCC scores in 2017 - see below – add more reason to question its higher quality rating. [Preliminary 2017 SPF ratings show that all three of Pueblo’s innovation middle schools—Roncalli, Risley, and Pueblo Academy of the Arts—have been rated on Turnaround.]

As Pueblo 60 was the first to make the baffling decision to grant more freedom to three of their chronically low-performing schools, we now have considerable data to see how it is going—including three years of PARCC scores.  Can anyone look at the growth and achievement results and argue that being an innovation school has proved beneficial?

GROWTH

Not one of Pueblo’s first three innovation schools demonstrated growth of even 40% on either subject.

Chalkbeat’s data base for 2017 Growth Scores includes this explanation: “Student growth measures how much groups of students learn during a year compared to students who scored similarly to them on previous state tests. The state’s average is also 50, which represents about a year’s worth of learning.”

              2017 – PARCC – school’s growth percentile scores
School
ELA
MATH
Roncalli STEM Academy
37
30
Pueblo Academy of the Arts
34
31
Risley International Academy
33
29.5

ACHIEVEMENT

D indicates a decline from 2016 for that grade in that assessment. Between 2016 and 2017 scores declined in most categories at Roncalli STEM Academy and Pueblo Academy of the Arts. Risley’s scores remain among the lowest in the state for any middle school.

Roncalli STEM Academy
ELA – % meeting expectations
Grade
2015
2016
2017

6
7.9
13.2
8.9
D
7
9.9
15.3
11.2
D
8
9.2*
15
15.2


MATH – % meeting expectations
Grade
2015
2016
2017

COMMENT
6
5.6
7.9
6.3
D
2015-2017 – less than 10% meet expectations in all grades
7
3.8*
9.8
NA**

8
3.8
5.3
4.1
D
**NA – fewer than 16 students

Pueblo Academy of the Arts
Grade
2015
2016
2017

COMMENT
6
19.4
17.2
19.5


7
32.8
28.5
13.1
D
Declined 19.7 % pts since 2015
8
18.8
33.0
26.9
D


MATH – % meeting expectations
Grade
2015
2016
2017

COMMENT
6
12.5
7.4
10.7

2016 & 2017 – less than 11% meet expectations in all grades
7
7.3
9.1
4.7
D
8
3.8
10.3
9.0
D

Risley International Academy
ELA – % meeting expectations
Grade
2015
2016
2017

COMMENT
6
15.2
7
13.2

2017 lower than 2015
7
31.2
18.9
9.7
D
Declined 21.5 % pts since 2015
8
10.1
14.8
15.5



MATH – % meeting expectations
Grade
2015
2016
2017

COMMENT
6
15.6
4.3
7.0

2016 & 2017 – less than 9% meet expectations in all grades
7
8.3
8.8
NA*

8
7.9
6.3
6.4

*NA – fewer than 16 students

**
And now a quick check on Pueblo 60’s newest schools in the district’s Innovation Zone, after year one.

2017 – PARCC – school’s growth percentile scores[viii]
School
ELA
MATH
Benjamin Franklin Elementary
31.5
41.5
Irving Elementary
51
48
Minnequa Elementary
36
28

At the three schools, 3rd grade PARCC scores improved. However, most 4th and 5th grade PARCC scores were lower in 2017 than in 2016. [Preliminary 2017 SPF ratings show the strong rating for Irving in 2016 continued in 2017 – on Performance; Benjamin Franklin’s rating dropped to Improvement; Minnequa’s rating continued on Turnaround.]


Addendum E – Aurora

Boston K-8, Crawford, & Paris added to innovation zone: “major win for Munn’s reform agenda”

Over a year ago, Aurora followed the course Pueblo 60 had taken:

AURORA — Three of Aurora’s most at-risk schools are closer to dramatic overhauls after the school board Tuesday approved plans to set the schools free of some district and state rules.
The schools will have greater autonomy over their budgets, hiring and curriculum if the State Board of Education signs off on the plans later this spring — the final step in a year-long process.
The unanimous vote to provide some of its school greater flexibility by the Aurora Public Schools Board of Education is a major shift for the state’s fifth largest school district, which has trailed the state in student achievement scores on annual standardized tests and graduation rates.
The board approved the plans without discussion. (Chalkbeat Colorado, March 2016)[ix]

In May of 2016 the state board told APS - OK, fine, go ahead.  Chalkbeat called it “a major win for Aurora Superintendent Rico Munn’s school reform agenda.[x]

PARCC results

In its story last month on 2017 PARCC results for APS, Chalkbeat Colorado stated:

Aurora’s five innovation zone schools, the biggest reform superintendent Munn has rolled out, saw mixed results. Last fall, the five schools each started working on plans the district and state approved giving them flexibility from some district or union rules and state laws.

Munn told Chalkbeat the “schools had only started working on the changes in their innovation plans months before students took these tests and said district officials aren’t yet attributing the results, negative or positive, to the reforms.”[xi] 

In its overview on the state results, Chalkbeat also noted that “among the state’s ten largest school districts, Aurora continued to post the lowest scores. For example, only 25 percent of fourth graders in the 41,000-student district met the state’s expectations on the English test.[xii] (Bold mine)

OK, too early for cause and effect.  Those of us who question this strategy for our lowest-performing schools would simply make three points to the State Board of Education and the Aurora School Board, for grades 3-5 at the three innovation schools serving those grades, Boston K-8, Crawford, and Paris:

1)      Only 25% of 4th graders” for the entire district – that is almost good news compared to the scores at the three schools.  
Fourth grade on English Language Arts – largely unchanged from the year before:
Crawford Elementary – 13.7% meeting expectations
Paris Elementary - 11.3% meeting expectations
Boston K-8: too few students to include 

2)      Overall at the three schools, for both ELA and Math results in 2017, we do not see one grade or subject matter where 17% of the students were meeting expectations.

3)      Even more troubling: in Math, the growth scores (MPG) were below 40% for all three schools.[xiii]

Data which cries out for help. Schools, teachers -- and students too, if they could understand all this–might be pleading as well.  Help. Support.  Intervention.  And not the fool’s errand of more autonomy.






[iv] https://www.chalkbeat.org/posts/co/2017/05/10/state-board-finalizes-path-for-aurora-central-hope-online-schools/ - “The proposal for Aurora Central is to continue following a plan to give the school more autonomy while getting help from an outside company.”
[vi] http://www.cde.state.co.us/accountability/performanceframeworkresults - In releasing this file last week, CDE reminded us that, as “these data are PRELIMINARY and not final, please use it with caution.” I include several findings where, though not final, the 2017 ratings could, in the end, support my argument.