Monday, February 16, 2015

AV#126-AP Results - What the Colorado Education Initiative Won’t Tell You

Another View #126                                                                                                         Peter Huidekoper, Jr.
Feb. 3, 2015

AP Results - What the Colorado Education Initiative Won’t Tell You

The Colorado Education Initiative has made claims about the success of its AP Initiative that it has not supported. As I wrote last year (AV#114-Questions continue on rationale for more AP classes in our lowest-performing high schools – 6/6/2014), it claimed a “70% increase in Advanced Placement scores … during the 2012-13 school year.” However, the information it made available showed the increase of those achieving qualifying scores that year was 33% (from 941 to 1,125).  When asked for information to show how students performed in its 10 “Legacy Schools” (cohort 1) in 2013, CEI (at the time, known as the Colorado Legacy Foundation)—after assuring me, over the span of four months, that it would do so, told me it would not make those results available to me. (End note #1)

What the CEI Initiative won’t tell you, but DPS will – p.2

Again this past November, when its press release announced “Advanced Placement Program Celebrates Record Gains,” asserting a 73% increase AP scores for the 10 schools that began implementation of the Colorado Legacy Schools initiative in the 2013-2014 school year (cohort 2), it was not accompanied by any report supporting such a boast (http://www.coloradoedinitiative.org/wp-content/uploads/2014/11/Press-Release-Nov-20-event-FINAL.pdf).  Greg Hessee, director of the Colorado Legacy Schools Program at CEI, was quoted by Nelson Garcia of channel 9 as making an even broader claim—for he appeared to speak of not just those 10 new schools, but all 23 schools in CEI’s work: “At the schools where the program currently exists in Colorado, he says 73 percent more students took AP courses and passed the exam for college credit” (http://www.9news.com/story/news/education/2014/11/20/non-profit-program-expands-access-to-ap-courses/70012630/). (To his credit, Garcia included two key words: “he says.”)

Go to CEI’s website and—incredibly, here in the winter of 2015—the most recent scores listed come from 2011 (yes, 2011)—with estimates for 2013 and 2014 in those 10 schools in Cohort 1 (http://www.coloradoedinitiative.org/wp-content/uploads/2013/04/Legacy-Schools-One-pager.pdf ).   Equally unhelpful—see “Results” at http://www.coloradoedinitiative.org/our-work/colorado-legacy-schools/results/ : just a graph of “first year
Copies of my two previous newsletters--#95
and #114-referred to here, are available at
AV#95 - Mismatch-adding more AP… (23 pages)
AV#114 - expanding AP in low-performing… (10 pages)
In #126 I try not to restate evidence presented there.
growth.”  This includes the impressive increase at three Colorado Springs-area high schools serving military families that CEI sent me last year: 48 qualifying scores in 2011 jumped to 256 qualifying scores in 2012.  Still, no specifics on its 10 new schools in 2012-13 (cohort 1). And not a word on the 10 newest schools added in 2013-14 (cohort 2). 

Fortunately, Denver Public Schools does release the data, school by school.  With the facts now available—proving how far off its earlier estimates turned out to be—CEI would do well to take down its “projections” for 2013 and 2014. 

After meeting CEI’s resistance last spring, I wrote AV#114—and waited. A new executive director was hired this winter.  Recently I asked her to take a look at my newsletter of last June, and at new questions related to the November 2014 assertion of a “73% increase.” The response: “We stand behind the data.”  

My argument all along – in my first piece on this issue (AV#95 - Mismatch - Adding more AP classes in low-performing high schools - Why the push to expand AP classes in schools where so many students fail to achieve qualifying scores? – 3/26/2013) and last year – has been that the AP Initiative is an inappropriate choice for schools where most students perform below grade level.  It does not address a much bigger problem. Abraham Lincoln High School in Denver has been my prime example—but it is not alone (End Note #2). In AV#95 I reported that Lincoln’s AP scores (4 out of 77 passed AP English tests in 2012) indicated it was one of several high schools where the AP expansion by Denver Public Schools (“Advanced coursework is pushed in DPS,” Denver Post, Aug.21, 2009) made little sense.  In #AV114 I included the dismal AP pass rate at Lincoln before the first year of CEI’s AP Initiative (2012)—24%, and after (2013)—again, 24%.

CEI told me last April that specific information is not available due to confidentiality agreements with the schools.  But note CEI’s bold assertions about “working to create a culture where using data to improve public education is everyone’s job”; “we believe in sharing what we learn”; “we act as the innovation and R & D partner for CDE” (End Note #3).  Hardly consistent with how it has reported on its AP Initiative. As what we do know fails to match what CEI has told us, the burden of proof is on CEI to open its books.

Last May CEI completed its second year with its AP Initiative at Lincoln. Here is what DPS reports.

What the Colorado Education Initiative won’t tell you, but DPS will

For five years now Denver Public Schools has presented the AP results at its high schools. (End note #4 - overall results.) Details at: http://testing.dpsk12.org/reseach_eval/reports/test_results/AP/AP_results.htm. There we see this added note: “Scores not reported for groups with fewer than 16 students.”  No “personally identifiable information.”  Understood.  Otherwise, DPS makes the results public, school by school, test by test.

Abraham Lincoln High School
AP Tests passed – from DPS Accountability, Research & Evaluation

2012 – BEFORE CEI AP INITIATIVE
2013 – First year of CEI AP INITIATIVE
2014 – 2nd year of CEI AP INITIATIVE

N tested
N passed
%
N tested
N passed
%
N tested
N passed
%
Biology
8
*
*
8
*
*
13
*
*
Calculus AB
29
2
7%
26
1
4%
35
13
37%
Chemistry
0
*
*
11
*
*
9
*
*
Computer Science A
13


11
*
*



Eng. Language & Composition
77
4
5%
75
3
4%
86
3
3%
Eng. Literature & Composition
37
4
11%
49
2
4%
37
0
0%
Environmental Science
0
*
*



13
*
*
Physics B
24
0
0%
37
1
3%
18
3
17%
Statistics






7
*
*

167
Total of tests taken where # passed is available
10
6%
187
Total of tests taken where # passed is available
7
3%
176
Total of tests taken where # passed is available
19
11%
* Scores not reported for groups with fewer than 16 students.

What is a “good” passing rate on AP tests? See Addendum B. National average – 55%        Colorado -  59.5%
Yes, 0 out of 37 passed the English Literature AP, and 3 out of 86 passed the English Language AP. In English on the AP English exams (both Language and Literature), the number taking the test has climbed from 2012, but the number passing has declined from       
8 to 5 to 3.  A 3% passing rate.  (Bold mine.)

CEI’s work focuses on MSE (math, science, and English) courses, which I assume includes the subjects above (the DPS website also lists scores for geography, history, Spanish, etc.). At Abraham Lincoln, for the four MSE courses where the numbers were large enough to report publicly, out of 176 tests taken, 19 received a 3 or better. A passing rate of 11%.  (19 3’s vs. 6 3’s in 2013; I wonder – could this be part of CEI’s

     PERSONAL POINT:  Why my extra attention to AP English?  Because I taught it.  I have a sense of the difficulty.  And I now help a few seniors at Rangeview High with their work in AP Literature. Great plays and novels.  Crime and Punishment this winter. I re-read it last month. Amazing, but a real challenge; not for everyone. As I wrote in AV#95, when I taught at a college preparatory school 30 years ago, the AP English class was not seen as appropriate for all our seniors.  I’m sure my own high school—where I did not qualify for Honors English senior year—would not have placed me in such a class.  Not for everyone.
happy news of a “73%” increase?)

If my categories are not what CEI calls its “MSE classes,” let’s hope CEI will produce its own report on which classes it does count. So far, with no such data available, I give you—thanks to DPS—a rough idea of how wrong CEI’s estimates proved to be, and how absurd it is that this is what CEI tells us in February of 2015.

CEI (still) lists 40* Lincoln students projected to pass AP MSE Exams in 2013.  DPS reports 6 passed.
CEI (still) lists 44* Lincoln students projected to pass AP MSE Exams in 2014.  DPS reports 19 passed.

By now Denver Public Schools and its principals must question if more AP classes—which is no doubt an excellent idea where a good number of students demonstrate they are ready for such college level work but have had not access to such an opportunity—is what a school like Abraham Lincoln needs.  Or a school like Bruce Randolph. Though Bruce Randolph High School is not participating in CEI’s AP Initiative, its AP scores on everything except the Spanish Language AP led me to write last June, “5 out of 126 passed in 2013. What’s the point of that?” Here is an update, with 2014 results.  Again, abysmal.  Why go on?

                Bruce Randolph – AP Results – WITHOUT Spanish Language scores (in 2014, AP Lit, AP Language, and US History)

                                     2009-10
     2010-11
     2011-12
       2012-13
      2013-14
# Tested                                     142                                              
        110
           90
         126
           106
# Passed                                      1
          2
            1
             5
               4
% Passed                                     1%                                       
          2%
            1%
             4%
               4%

Sufficient evidence, wouldn’t you agree, to say: STOP! Far better to develop curriculum and instruction that meets these juniors and seniors where they are–most of them not yet performing at grade level–and to help them make as much progress as possible. Classes where they find success, improve their skills, and—one hopes— graduate without needing remediation in reading, writing, and math once they enter college. 

                   How, GOOD – no, GREAT! –
to think of Jim Collins presenting to the Colorado Association of School  Boards in Colorado Springs this past fall. Nice, too, to see on CASB’s website: Explore these helpful tools on the Jim Collins website, among them “Jim’s 12 Questions,” number 12 of which reads: “What should be on our Stop Doing list?”
    School boards, superintendents, and principals would do well to insist that CEI make public its data on the effectiveness of its AP Initiative at Abraham Lincoln, Aurora Central, and other low-performing high schools.  If the data reflects what I have found, here is a perfect example of what school leaders should tell CEI to Stop Doing.
“It’s a matter of what’s a good fit for kids” – Ken Seeley

This is not lowering expectations. It is being realistic about what is a good fit.

And it is taking the results of the AP Initiative in schools like Abraham Lincoln and making them public, so that we can see what is and is not working—and make better choices to serve students in such low-performing schools.   It is past time for the Colorado Education Initiative to come forth with honest information on the results of its AP work.  The students at Lincoln—and schools like it—deserve better than this.


Another View, a newsletter by Peter Huidekoper, represents his own opinion and is not intended to represent the
view of any organization he is associated with.  Comments are welcome. 303-757-1225 / peterhdkpr@gmail.com



End Notes
1.      “… we have agreements with the schools we fund regarding release of data.  We cannot and will not violate the relationships we have with our school and district partners, nor do I believe you would want us to.” Email from Dr. Helayne Jones, President and CEO of Colorado Legacy Foundation, April 22, 2014.

2.      School Performance Framework
These four high schools in CEI’s AP Initiative are on Priority Improvement or Improvement. Their Academic Achievement Rating indicates the focus should be on grade-level skills—not more college level (AP) classes.


District
Performance Indicator
Rating
% Points Earned out of Possible Points
Abraham Lincoln
DPS

Academic Achievement
Does Not Meet
25%
Aurora Central
Aurora
Does Not Meet
25%
Northglenn
Adams 12
Does Not Meet
33.3%
Northridge
Greeley 6
Does Not Meet
25%

Not in CEI’s AP Initiative, but also on Improvement:

Bruce Randolph
DPS
Academic Achievement
Does Not Meet
25%

3.      On data, sharing, and its R& D role  -  what the Colorado Education Initiative states

A)     On the importance of using data: Home page, “Who We Are”
“CEI is working to create a culture where using data to improve public education is everyone’s job, and data-driven decision making becomes ‘the new norm’ to improve student outcomes and to provide a professional and rewarding environment for educators.”
B)     On “sharing what we learn”
“WE BELIEVE IN SHARING WHAT WE LEARN - Our role as a thought partner and partner with the Colorado Department of Education and as a catalyst for innovation in schools and districts has garnered national attention and interest from several states considering the creation of similar organizations. We are excited to find ways to pass on the good work we are doing with others.”

C)    On partnerships and sharing knowledge
“We work in partnership with the Colorado Department of Education and education, business and policy partners across the state to accelerate bold improvement in student achievement through innovation, collaboration and capacity building…. By sharing knowledge and collaborating with diverse stakeholders, we are identifying a shared vision and pathway for collective impact in public education.”

D)     CDE’s R & D partner
Today, we act as the innovation and R & D partner for CDE.”

Over the past few years CDE’s Turnaround Office has awarded large federal grants (School Improvement Grants) to several of the state’s lowest-performing schools like Bruce Randolph, Montbello, and West in Denver and Aurora Central High School in APS. (AV#121-After$63 million - Oct. 2014)  All of them have expanded their AP offerings.  Has it helped?  CDE’s Turnaround Office must be eager to know if adding college-level classes at schools with some of the state’s lowest achievement scores makes sense.

This is exactly why CEI should show its results, school by school.  Its “R & D partner” is distributing millions of federal dollars and the results have been disheartening.  CEI can show us if its work makes any sense in a school like Aurora Central High, now receiving almost $2.4 in federal funds over three years for “transformation.” Here is a school that is entering its 5th year on Priority Improvement or Turnaround. Graduation rate last year: 46.2% (state average: 77.3%) (http://co.chalkbeat.org/find-your-schools-2014-graduation-rate/). Its ACT Composite Scores in 2014: 15.2 (state average: 20.3). http://www.cde.state.co.us/assessment/coact-dataandresults Math proficiency on TCAP in 2014—10.2% (state: 36.5%).  Writing proficiency—19.2% (state: 51%). (https://edx.cde.state.co.us/SchoolView/DataCenter/reports.jspx?_afrWindowMode=0&_afrLoop=4095624656562705&_adf.ctrl-state=g91gqbpo4_4).  Why, I ask, would CDE send one dime to such a school – to expand its AP classes?

CDE should know better. Aurora Public Schools should know better. As for Aurora Central—well, cue up the Steve Miller Band: “go on take the money and run.

We keep hearing about finding and replicating strategies that work.  To that end it is critical to have data on the AP Initiative for the state and districts and principals to know if it should keep pushing these courses, or not, in schools like Lincoln, Aurora Central, and others. CEI could help.  Give us the results.

4.      From DPS - Assessment, Research & Evaluation – 2014 AP Tests Passed by School


2014
2014

# of tests passed
% passed
STRIVE Prep – SMART Academy
112
81%
Denver School of Science & Technology: Stapleton
190
73%
Denver School of Science & Technology: Green Valley Ranch
39
58%
East
998
57%
Denver School of the Arts
265
54%
Denver Center  for International Studies
140
53%
KIPP Denver Collegiate
85
48%
Thomas Jefferson
201
35%
George Washington
148
31%
South
152
31%
Montbello
32
30%
In the following seven schools, less than 30% of tests taken scored at 3 or above
Bruce Randolph H.S. (6-12)*
36
26%
North
55
24%
Abraham Lincoln
107
23%
DCIS at Montbello
21
22%
John Kennedy
36
21%
Martin Luther King Early College
55
13%
Manual H.S.
2
2%


Addendum A – If Thomas Jefferson High (cohort 2) is one of CEI’s real successes…?

CEI’s press release (Nov. 20, 2014) stated: 
“The AP teachers and students at Thomas Jefferson High School far exceeded our expectations last year. The students’ 153 percent increase in qualifying math, science, and English [MSE] AP scores represents one of the largest improvements in the entire state,” said Gregg Fleisher, Chief Academic Officer for the National Math and Science Initiative. “Through our partner, The Colorado Education Initiative, we are proud to support Thomas Jefferson and the state of Colorado to ensure that more students have the opportunity to pursue majors and careers in STEM fields.”

What DPS will tell you …
Thomas Jefferson High School - AP Tests passed – from DPS Accountability, Research & Evaluation


2012 - BEFORE CEI AP INITIATIVE
2013 - BEFORE CEI AP INITIATIVE
2014 – First year of CEI AP INITIATIVE

N tested
N passed
%
N tested
N passed
%
N tested
N passed
%
Biology
13
*
*



18
13
72%
Calculus AB
24
7
29
25
3
12%
13
*
*
Chemistry
0
*
*



11
*
*
Computer Science A
0
*
*






Eng. Language & Composition
65
19
29
76
10
13%
104
39
38%
Eng. Language & Literature
34
8
24
21
8
38%
40
10
25%
Environmental Science
0
*
*



62
15
24%
Physics B
13
*
*
20
12
60%
12
*
*
Statistics
0
*
*



7
*
*

123
Total of tests taken where # passed is available
34
28%
142
Total of tests taken where # passed is available
33
23%
267
Total of tests taken where # passed is available
77
29%
*Where 15 or fewer students took the test in that subject, DPS does not release the number passing.

So “one of the largest improvements in the entire state” saw 125 more students take these MSE AP tests in 2014 than in 2013, with 44 more passing (77) than the previous year (33). Kudos to Thomas Jefferson’s AP Biology teacher and students: 72% passed!  But on other MSE tests (English and Environmental Science) only 31% passed (64 out of 206). Overall DPS figures show the passing rate at Thomas Jefferson last year, on all the MSE AP tests, to be 29%—just 1% higher than in 2012—28%.

I can see some value in expanding AP opportunities at Thomas Jefferson. When the number taking and the percentage passing AP tests both climb—as DPS shows they did at TJ, George Washington, and KIPP Denver Collegiate from 2013 to 2014—the evidence suggests: it’s worth a try. But the data above raises doubts about the other CEI schools.  Because if Thomas Jefferson is one of CEI’s best stories for 2014….


Addendum B – What is a “good” passing rate?

What is a “good enough” percentage of students passing on AP tests? Do we expect 100% of the students taking the most difficult Physics AP to pass, as we saw at Denver School of Science and Technology last year? (CONGRATULATIONS TO DSST PHYSICS TEACHER HILARY WALKER AND HER STUDENTS! We would do well to keep in mind Hilary’s comment to me on the great results: “the preparation really began freshman year.”)

No, but we ought to take into account the national average: 55% of total AP exams received a 3 or better in 2013.  In Colorado, the average for AP passing scores in 2013 was 59.5%. 
On the AP English Literature exam, across the county, 55% received a 3 or better in both 2014 and 2013.

In Colorado, 63.1% passed this exam in 2013. (Details, supporting documents - Addendum C.)

When less than one third of the students in a school who take a test pass (a score of 3 or better), it is fair to ask if this is a good fit in this building.  Are expectations high enough when enrolling students in the AP classes? Any wonder that we hear of the curriculum being watered down to adjust for the skills of the students, resulting in a course that is “AP” in name only?  In such cases, no wonder so few pass the exam.

Centennial High – one of CEI’s “Legacy Schools” – cohort 1

It is also disturbing that some high schools think “better AP results” might earn them a higher rating on the state’s performance rating. Like Abraham Lincoln, Centennial High in Pueblo is one of the schools in cohort 1 of the CEI AP Initiative. Centennial recently asked the Colorado Department of Education for a higher rating due to its AP results.  CDE’s denial indicated it was not impressed by a passing rate near 44%.  One assumes CDE would find passing rates below 30%--one-half the state average on AP exams—even more disappointing.

From “2014 District Accreditation and School Plan Type Assignments –
Request to Reconsideration Summary,” page 71.
PUEBLO CITY 60 – Centennial High School

“In its request, the district proposed that CDE consider the results of supplemental assessment results from annual Advanced Placement (AP) exams as evidence that Centennial High School should be assigned a higher performance rating. The district argued that the school’s improved performance on the AP exams from 2010 through 2014 reflected reform efforts that have been aimed at implementing a more tightly aligned academic framework. The district suggested that this supplemental data should carry enough weight to warrant a higher rating on the Post-Secondary and Workforce Readiness (PWR) indicator, which would in turn result in a higher overall percentage of points, and would thus justify a Performance plan type….”

Reason for Denial (p. 71)
“CDE determined that although the performance of students taking the AP exam has improved since  2010, the percentage of Centennial students scoring three or higher on an AP exam remained 18 percentage points below the state average of 62.2%. For this reason, the review committee determined that the AP results did not provide sufficient evidence of improved student performance to approve the request to move the school to Performance.” (Bold mine) http://www.cde.state.co.us/accountability/2014districtandschoolrequeststoreconsider

Addendum C

The 10th Annual AP Report to the Nation

UNITED STATES – 2013 - Total AP Exams – 1,824,503
Total AP Exams Scoring 3 or better  -   1,000,135  = 55%   (page 7)

COLORADO – 2013 -  Total AP  Exams – 57,314   
% of Exams Scoring 3 or better = 59.5%   (Figure 4, page 13)

________________________________________________________________________________________
AP English Literature and Composition Exam
Nationally, both of the last two years, 55% of students scored a 3 or better.

UNITED STATES – 2014 – 55%
EXAM Score
N
% AT
5
30,422
7.7%
4
70,602
17.8%
3
117,748
29.6%
2
131,210
33.0%
1
47,495
11.9%
Number of students
397,477

3 or Higher / %
218,772
55%

UNITED STATES -  2013 -   55%


In COLORADO, on the AP English Literature and Composition exam, 63.1% scored a 3 or better in 2013.
EXAM Score

% AT
5

63.1% pass
(3 or higher)
7.8%
4
20.5%
3
34.8%
2

31.2%
1

5.8%

1 comment: