Wednesday, November 14, 2018

AV#186 - AP results shows failure of CEI effort in a low-performing high school




When no one listens the first five times you make your point (see box), you know it is time to take a different tack. So a simple request: please look at page 3, the AP scores for one Colorado high school.

AV#95 – “Mismatch - Adding more AP classes in low-performing high schools” (March 2013); AV#114; AV#126; AV#134; “OPEN LETTER TO: Denver School Board and Staff” (March 2017)
The Colorado Education Initiative, with its Colorado Legacy Schools (CLS), has not made public any Advanced Placement scores for their 47[i] schools for three years (last mention of results on CEI’s website is from November 2015[ii]). School districts rarely publish their scores. Aurora Public Schools stopped presenting school-by-school results, as it had done for six straight years, after 2013[iii]; Denver Public Schools no longer provides a breakdown of the number tested and passed, by course, for each high school, as it had up until 2016.[iv]  Three cheers, then, (and my deep thanks) to the Harrison School District for providing the AP results for the past six academic years, 2013 to 2018, at Sierra High. 

If CEI shows different results than what Harrison reports, I trust it will make them public.  Just as we should be glad to see the details of AP results at two APS and DPS low-performing high schools, Abraham Lincoln and Aurora Central, schools like Sierra High that CEI never should have included in its AP push.  But again – ever since I included their AP results in previous newsletters – we see little to nothing.[v]

CEI states:
Our Colorado Legacy Schools (CLS) initiative works with 47 high schools across the state to dramatically increase the number and diversity of Colorado high school students who are succeeding in AP math, science, and English courses — especially students typically underrepresented in AP courses such as females, and students of poverty and color.”[vi]

A “dramatic increase …”  As I have shown before, this never happened at Abraham Lincoln and Aurora Central. I had hoped anyone who studied the alarmingly poor results of CEI’s effort there would have agreed.  And – while not a clear cause and effect, but a factor - the schools continued to falter.  Are we able to admit our mistakes? Can we see when our desire “to help” is misguided, even harmful?

At its website, we read: “Colorado Legacy Schools - College success begins with high school success.”[vii]



School Performance Framework
Percentage Points Earned

2012
Prior to CEI AP effort
2013
CEI – Yr 1
2014
CEI – Yr 2
2016*
After CEI AP support
2017

2018
(Prelim)
2018
(Prelim)
Improvement
Improvement
Priority Improvement - Yr 1
Priority Improvement - Yr 2
Priority Improvement - Yr 3
Priority Improvement - Yr 4
41.8%
Aurora Central High School (APS)
Priority Improvement
- Yr 3
Priority Improvement - Yr 4
Priority Improvement - Yr 5
TURNAROUND
- Yr 6
Priority Improvement - Yr 7
Priority Improvement - Yr 8
37.4%
*2015 was the third year of the CEI AP Initiative at these two schools; due to a shift in assessments, no SPF was produced that year.

So much for success. But that is old news.  CEI just completed the third year of its AP Initiative at Sierra High.  What follows provides the AP results there—data reported for 2013-14, prior to the Legacy Schools initiative, and then, over the three years of CEI’s support - 2015-16 to 2017-18.  The results show that the work at Sierra High not only did not “dramatically increase” the number of students there “succeeding in AP math, science, and English”—if success means a passing score. In those subjects, the number declined.

The data convinces me that CEI’s AP Initiative at Sierra failed. The results support the argument I have made for years: with our lowest-performing high schools, it is a mistake to emphasize placing many more students in AP classes.  Classes designed to offer college-level work are rarely a good match for schools where the far majority of students are not reading, writing, or doing math at grade level.  Will anyone argue the CEI effort helped Sierra High School improve?




School Performance Framework[viii]


2014
Prior to CEI effort
Percentage Points Earned
2016
CEI – Yr 1
Percentage Points Earned
2017
CEI – Yr 2
Percentage Points Earned
2018 (Preliminary)
CEI – Yr 3
Percentage Points Earned
Sierra High School
(Harrison School District)
Performance



63.6%
Improvement-decreased due to low participation

55.5%
Improvement – low participation


44.7%
Priority Improvement -decreased due to
low participation
44.5%

(See Addendum A for details behind these declining SPF ratings during the first two years of CEI’s AP Initiative.)

The Harrison School District sent me the number of students at Sierra High taking and passing each of these AP classes, and the pass rate per course.  My overall tally (next page) excludes the good story about the success Sierra students have had on the Spanish Language and Culture class.  It was already—prior to the CEI effort—the one class that stood out for having a majority of the students pass the AP exam – 58.8%, in fact (10 out of 17 in 2015).  It was great to see the 2017 results: 15 out of 17 students passing this Spanish AP exam (88.2%), and the 2018 scores: 17 out of 20 passing in 2018 (88%).  However, such passing rates have been an anomaly in a school where the percentage scoring a 3 or better is otherwise terribly low.  So it seems important to note the distinction – the TOTAL WITH and TOTAL WITHOUT the impressive passing rates for the Spanish AP – in order to capture if the AP push proved effective during the three years Sierra was part of the CEI project. 

Again, the data here comes from the Harrison School District.

First, my summary of the chart that follows.

SIERRA HIGH SCHOOL – AP results - Summary


 (before CEI support)
2016 (year 1 CEI support)
2017 (year 2 CEI support)
2018 (year 3 CEI support)








N
Pass Rate
# Passing
N
Pass Rate
# Passing
N
Pass Rate
# Passing
N
Pass Rate
# Passing
135
21.5%
29
328
8.8%
29
255
8.6%
22
212
13.2%
28


Now, the details. Look closely at the results in English, Math, and Science courses, the focus of CEI’s work.

SIERRA HIGH SCHOOL - AP results


2015
 (before CEI support)
2016 (yr 1 CEI support)
2017 (yr 2 CEI support)
2018 (yr 3 CEI support)


N
Pass Rate
# Passing
N
Pass Rate
# Passing
N
Pass Rate
# Passing
N
Pass Rate
# Passing
Biology
NA




31
9.7%
3
11
*


NA




Calculus AB
16
31.3%
5
60
11.7%
7
25
16%
4
19
21.1%
4
Calculus BC
NA




9
*


12
*


8
*


Calculus BC:
AB Subscore
NA




9
*


12
*


8
*


Chemistry
13
*

21
4.8%
1
NA


NA


Computer Science Principles
NA


NA


8
*

NA


English Language & Composition
28
32.1%
9
52
5.8%
3
63
7.9%
5
NA


English Literature & Composition
26
26.9%
7
34
17.6%
6
38
5.3%
2
29
10.3%
3
Environmental Science
21
9.5%
2
22
0%
0
18
16.7%
3
27
7.4%
2
Human Geography
NA


25
12%
3
34
2.9%
1
40
2.5%
1
Physics I
NA


NA


6
*

NA


Psychology
17
11.8%
2
16
12.5%
2
15
*

61
24.6%
15
Spanish Language & Culture
17
58.8%
10
13
*

17
88.2%
15
20
85%
17
Statistics
9
*

10
*

11
*

9
*

Studio Art: 2-D Design Portfolio
NA


NA


NA


3
*

Studio Art: Drawing Portfolio
6
*

10
*

5
*

NA


U.S. Government & Politics
NA


15
*

NA


NA


U.S. History
NA


28
10.7%
3
35
2.9%
1
14
*

World History
27
14.8%
4
39
2.6%
1
43
14.06%
6
22
13.6%
3
TOTAL W/ SPANISH
180

39
394

290
341

37
260

45
TOTAL W/O SPANISH
135
21.5%
29
328
8.8%
29
255
8.6%
22
212
13.2%
28


A few highlights:

1. By year three of the CEI effort, Sierra High School had dropped seven AP courses (5 of them in the English, Math, and Science field – the focus of CEI’s Initiative):
  1. Biology*
  2. Chemistry**
  3. Computer Science Principles
  4. English Language Composition***
  5. Physics
  6. Studio Art: Drawing Portfolio
  7. U.S. Government & Politics

During the three years of CEI support, the passing rates on these seven courses, where shown, were shockingly low.  One assumes that was a factor in Sierra dropping them.

*Biology - 2016: 3 out of 31 passing.
**Chemistry – 2015 – not offered; 2016, year one with CEI: 21 students, one passed; course then dropped for 2017 and 2018.
***English Language & Composition – number taking grew from 2015 (28 passing) to 2016 (52) but the number passing declined, as did the percentage passing:  2016: 3/52  - 2017:  5/63  


2.  Growth in the number taking at times led to a dramatic drop in the percentage passing:

English Literature & Composition: number taking grew from 2015 during the three-year initiative, but the number and the percentage passing declined: 2015:  7/26     2016:  6/34   2017: 2/38     2018:  3/29

Human Geography: not offered prior to 2016, numbers taking grew from 2016 to 2018, yet numbers passing incredibly low:  2016: 3/25     2017: 1/34    2018:  1/40

Passing rates in several courses were low, then fell further:


AP EXAM
2015
 (before CEI support)
2016 (yr 1 CEI support)
2017 (yr 2 CEI support)
2018 (yr 3 CEI support)

N
Pass Rate
# Passing
N
Pass Rate
# Passing
N
Pass Rate
# Passing
N
Pass Rate
# Passing
English Literature and Composition
26
26.9%
7
34
17.6%
6
38
5.3%
2
29
10.3%
3
21
9.5%
2
22
0%
0
18
16.7%
3
27
7.4%
2
Human Geography
-
-
-
25
12%
3
34
2.9%
1
40
2.5%
1
World History
27
14.8%
4
39
2.6%
1
43
14.06%
6
22
13.6%
3


Yes, that really is a ZERO - 0 out of 22 students in 2016 - passing the Environmental Science exam. 

In four years, only 7 out of 88 students taking the Environmental Science exam received a passing score.  In three years, only 5 students out of 99 taking World History earned passing score. 
With such bleak passing rates, one would think Sierra might drop these AP courses as well.


Accountability. Transparency. Facts.

What would I like to see? No more opinions about how we can make our high schools more equitable by providing greater access to students who have previously not had the opportunity to take rigorous college-level classes. No more arguments about how the results at a school like Sierra High fail to take into account the benefits of merely taking a more rigorous courses, whether the student earns a passing rate or not.  We are familiar with such points. What we have not seen lately – from CEI, DPS, or APS – are the results of the AP expansion in our state’s lowest-performing high schools.  That is what I look for.   Once such data is made public – I have not lost hope – then we might have a discussion based on the facts.



Addendum A – Sierra High School

From Colorado's School Performance Frameworks

2014
Prior to CEI effort
Percentage points/ Rating
2016
CE- Yr1
Percentage points/ Rating
2017
CEI – Yr 2
Percentage points/ Rating
2018
CEI – Yr 3
Achievement
50% /
 Approaching
42.1% / Approaching
29.3% /
Does Not Meet

Growth
64.3% /
 Meets
64.4% /
Meets
48.9% / Approaching

PWR
68.8% /
Meets
56.9% / Approaching
54.2% / Approaching

CO- SAT – EBR (READING)


452.4 /
Does Not Meet
464
(STATE AVERAGE)


513.4
513
SAT – MATH


435.5 /
Does Not Meet
445
(STATE AVERAGE)


500.9
501
Data from School Performance Frameworks - 2014, 2016, 2017* – and SAT scores from 2018**
** https://www.cde.state.co.us/assessment/cosatdataandresults



[i] This total of 47 comes from CEI website - https://www.coloradoedinitiative.org/our-work/colorado-legacy-schools/; CEI’s website also shows another 8 schools (cohort 6) are half-way through their two-year participation in the CLS program(https://www.coloradoedinitiative.org/our-work/colorado-legacy-schools/colorado-legacy-schools-cls/), which would bring the total of Legacy schools to 55.
[ii] http://www.coloradoedinitiative.org/wp-content/uploads/2015/11/Final_2015CLSStateResults_Cohort3_20151117.pdf  And this only had details on ONE school. “Glenwood Springs High School (GSHS) in Roaring Fork School District is celebrating incredible first-year results in the program.” GSHS Principal Paul Freeman is quoted there as stating: “This school year, our student enrollment in AP courses jumped to 234 enrolled students and 312 students have already enrolled in AP courses for the 2016-2017 school year — we’re really excited about our results.” No data reported on any of those 11 schools in that cohort for years two and three; no data reported on ANY of the 3 schools in cohort 4 or the 10 schools in cohort 5, which have all completed their three-year participation in the CLS program.
[iii] From AV#137 (October 5, 2015) - “Up until 2013 Aurora Public Schools released results on the AP tests at its high schools.  But as of [a] year ago March, the APS Division of Accountability and Research told me ‘our office doesn’t do the AP report any longer.’” (NOTE: Back in 2014, AV#114 included six years of AP results at Aurora Central High -2007-8 to 2012-13). In 2017 I asked APS if this was still the policy: The response from Mya L. Martin-Glenn, Ph.D., Director, Assessment & Research, Division of Equity in Learning: “Yes, that is still the case. We do not post or release AP results at the high schools and we do not require the high schools to post that information.”
[iv]In 2016 DPS gave results for all STEM tests and all English tests but no longer gave a breakdown by specific tests/courses.  The past two years the public sees even less. After receiving the 2017-18 AP report from DPS in early November, I asked if there would be a breakdown of the results by schools and by the tests. The response from Danielle Johnson, Research and Strategy Manager: “It will not. This is the reporting structure we moved to using a couple years ago and the only report we produce internally and externally.”   
[v] The 2017-18 results present one line per school. It is intriguing, perhaps a first step away from the AP push, to see the number taking AP tests declining significantly at schools like Abraham Lincoln (493 in 2015, 219 last year); MLK Jr. (413 in 2014, only 36 last year); and under three digits at JFK (64), Noel Arts (43), and High Tech (30) in 2018.