With no ACT, no PARCC, & no proficiency benchmarks in our ESSA plan, is NAEP all we’ve got?

Mid-summer, and a chance to reflect: Gaining ground? Inching forward?

In several newsletters, I have challenged how districts, the state – heck, the nation – cheer higher graduation rates (#162); how the Colorado Education Initiative claims success in expanding Advanced Placement courses in dozens of high schools (#137, #114, and #95); and how Colorado Succeeds celebrates progress on the READ Act after one full year (#144). Each time, I am asking a basic question about how we measure success. In the cases above, it seems I measure progress differently from others.

A few thoughts here as to whether Colorado has any agreed upon benchmarks to measure progress. I conclude with a brief look at the long-term goals in Colorado’s ESSA State Plan and ask if this newest way to assess our progress is sound.

The Colorado State Board of Education also wonders

Are we banging our head against a brick wall?

All of us paying attention to public education ask if we see improvement, so I was glad to learn that members of the state school board wonder too.

On June 15, as the state board concluded its recent work reviewing and usually approving of the proposals from schools and districts on year 5 or 6 of the “accountability clock,” board member Steve Durham reflected on the big picture of public education.

Jason Glass on national achievement gains

The new superintendent in Jefferson County Schools offered a similar assessment, in 2015, after the first-year PARCC results were released.

“Nearly 15 years after the passage of No Child Left Behind, which ushered in this era of testing, big data and public shaming of so-called ‘low performing schools,’ … our national achievement gains have actually slowed and reversed.” http://www.vaildaily.com/opinion/editorials/vail-daily-column-not-much-learned-from-tests/ (Nov, 17, 2015)

“I think if there’s any evidence of progress in the last 50 years, progress - from a measurable perspective in terms of test scores and those obvious things - it has been almost nonexistent – for all the pain and suffering and all of the reforms - Dr. Schroeder (referring to state board chair Angelika Schroeder) and I have disagreed about this, she thinks there is some progress - but no one can characterize it as dramatic improvement.”

As Colorado recently decided to end our participation in the PARCC assessments, we will no longer be able to compare how our K-8 students perform against other states still using PARCC. Such comparisons had been among the goals behind joining one of the two consortiums, PARCC or SMARTER BALANCED, several years ago—as over 40 states did; with Colorado joining many other states withdrawing from PARCC, only 5 of the original 24 PARCC states will participate in 2017-18.

Our shifts from CSAP to PARCC to what-we-come up-with-next-year means that measuring and comparing student performance over time is, well, a work in progress. The Colorado Department of Education offers assurances that we will be able to compare how our students do—year to year—at least within our state. According to Chalkbeat Colorado:

“Colorado has already changed math and English testing twice in the past decade, making comparing past results extremely difficult — if not impossible. Officials say it won’t be the case now because this is essentially a contract change. However, more significant test changes may need to be considered after the state’s academic standards revision process is completed in 2018.” (http://www.chalkbeat.org/posts/co/2017/06/14/colorado-will-no-longer-give-parcc-english-and-math-tests-forging-its-own-path/).

A look back – CSAP and NAEP

Looking back 20 years, when CSAP was introduced, can we see progress? Yes - CSAP/TCAP scores rose; e.g. in 1997, 57% of Colorado’s 4^th graders were proficient or advanced in reading; by 2014, the percentage reached 67%. A closer look at 2005-2014, however–see Addendum A—shows only small gains over those final 10 years.

We can see improvement in individual schools, and certainly in specific districts like DPS (see A Plus Colorado’s: Start with the Facts, 2017 –page 4, “How Has DPS Performance Changed Over Time?” http://apluscolorado.org/wp-content/uploads/2017/05/DPS-Report-Website-Version.pdf?ref=1725). But as a state … what would we say? Especially now that PARCC results indicate that throughout the CSAP/TCAP years we set the bar for proficiency—let’s admit it—too low.

Without PARCC to compare with other states, is NAEP how we measure progress?

“NAEP, a.k.a. ‘the Nation’s Report Card,’ is the only nationally representative and continuing assessment of what America’s students know and can do in different subject areas. Its two major goals are to measure student achievement and to report change in performance over time. NAEP provides results for the nation as a whole and for the states separately.”

http://www.cde.state.co.us/assessment/conaep

A too-simple response to Steve Durham’s point would be to note the progress for Colorado students on the National Assessment of Educational Progress (see box) over 17 years.

NAEP scores for Colorado students - % at or above proficient

	1998	2009	2015	1990’s to 2015 - Scores up by
4^th grade Reading	33	40	39	6 points
8^th grade Reading	30	32	38	8 points
	1996	2009	2015
4^th grade Math	22	45	43	19 points
8^th grade Math	25	40	37	12 points

Gains that, as NAEP likes to say, are “statistically significant.” In math, up until 2009 anyway, impressive!

Then again, one looks more closely at NAEP scores over the last decade, especially since 2009, and wonders if Durham’s concern (progress seems “almost nonexistent”) isn’t well-taken:

	Colorado NAEP scores – at or above proficient					Gain ’07 to ’15
	2007	2009	2011	2013	2015
4^th grade Reading	36	40	39	41	39	+3
8^th grade Reading	35	32	40	40	38	+3
4^th grade Math	41	45	47	50	43	+2
8^th grade Math	37	40	43	42	37	-

More disheartening is a look at where we said we would be on NAEP by 2015. In Colorado’s Race to the Top application (2009) to the U.S. Department of Education, we set a series of goals for CSAP and NAEP scores. Our application envisioned Colorado students performing as well or better on NAEP than Massachusetts students by 2015 - see Addendum B from AV#63 - Colorado’s RTTT Goal: … How credible …? (July 2010). Perhaps it was a good idea to use the highest-performing state as our marker, but I described the goals as “illusory,” “impossible,” and “unreachable.” I recall an unpleasant conversation with a CDE leader back then, who explained that the goals were “aspirational.”

No need to be a prophet to see we would fall short, as the 2015 results, below, make clear. True, we did not win that RTTT grant; big bucks from the federal government might have helped. By 2015 our students averaged 10% points below Massachusetts students on the four tests, and over six years the gap between Colorado and Massachusetts students widened for 4^th grade reading and 8^th grade math. On those two assessments, Colorado’s scores declined between 2009 and 2015: from 40 to 39 on 4^th grade reading; from 40 to 37 on 8^th grade math.

NAEP scores - % at or above proficient

	Colorado					Massachusetts	Gap btw/ Mass & CO 2009	Gap btw/ Mass & CO 2015
	2007	2009	2011	2013	2015	2015	Gap btw/ Mass & CO 2009	Gap btw/ Mass & CO 2015
4^th grade Reading	36	40	39	41	39	49	-7	-10
8^th grade Reading	35	32	40	40	38	45	-11	-7
4^th grade Math	41	45	47	50	43	54	-12	-9
8^th grade Math	37	40	43	42	37	51	-12	-14

Reality check

RTTT goal for 4^th grade reading – climb from 40% proficient in 2009 to 60% proficient.

Instead, we declined by 1% - 39% proficient.

RTTT goal for 8^th grade reading – climb from 32% proficient in 2009 to 52% proficient.

Instead, 8^th grade scores increased by just 6% points to 38% proficient.

RTTT goal for 4^th grade math – climb from 45% proficient in 2009 to 65% proficient.

Instead, we declined by 2% - 43% proficient.

RTTT goal for 8^th grade math – climb from 40% proficient in 2009 to 60% proficient.

Instead, we declined by 3% - 37% proficient.

And let’s not miss the larger point: on NAEP, in 2015, over 55% of Colorado students were NOT proficient on all four assessments.

Is NAEP our sole remaining benchmark to measure progress over time for K-12 schools in Colorado? Back in 1996, I used NAEP and ACT Colorado scores in a report for the Gates Family Foundation to measure progress, 1991-1995. As we shift from ACT to SAT for high school assessments, and as we wait to see how well 2018 assessments can be used to compare to the CMAS scores of the past three years, is NAEP the best we can do?

Some will say yes, especially as we now see NAEP’s scores, especially in reading, ring true – aligning so well with PARCC results. A bleaker picture than what CSAP scores had suggested.

2015 results - % at or above proficient	NAEP	PARCC
4^th grade Reading	39%	41.7%
8^th grade Reading	38%	40.9%

Others will disagree, reminding us that only 2,200 Colorado students take the NAEP reading test and another 2,300 the NAEP math test, in fewer than 100 Colorado schools.[i] So even if NAEP is a useful benchmark, it won’t do for accountability purposes. The tests are taken, according to CDE, by a “representative sample of students at the state level”; still, NAEP results do not give us information on how our students are doing, in our school, our district, and across our state.

CDE’s goals – clear, measurable, credible?

In June of 2016 the Colorado Department of Education presented four state-wide goals in its Performance Plan (http://www.cde.state.co.us/cdecomm/cdeperformanceplan). Goal #1 is to expand pre-school services. Fine, but there are no measures of student learning. Goal #4 (“Graduate ready”) speaks of raising graduation rates over 90% by 2019. In AV#162 I explained why I do not believe graduation rates tell us anything meaningful about academic achievement (“Higher graduation rates - Fake News”). Low ACT scores and high remediation rates speak volumes.

Perhaps Goals #2 and #3 can be measured, even though both are surely “aspirational”:

#2 – “Every student reads at or above grade level by the end of third grade.”

#3 – “Every student meets or exceeds standards.”

Consider #2. In its 2016 Performance Plan CDE included the dated information from the 2014 TCAP assessment (3^rd grade proficiency - 71.6%) and dated goals (“increase third grade reading proficiency on TCAP … with the goal of nearly 85% proficient by 2018”). It added: “With the new state assessment, the goals will be revised in alignment with our ESSA plan.”

Revised? How about repeal and replace! Two years of PARCC results tell us (have we really come to grips with this?) that the majority of 3^rd graders are not reading at grade-level.

3rd grade reading	PARCC- Met or Exceeded Expectations
2015	38.2%
2016	37.4%

In departing from PARCC, can we still use this baseline data? I hope so. If we can, will the state now give us (what is not in our ESSA plan—see next section), meaningful goals for 3^rd grade reading proficiency? How about … say, 40% proficient by 2018? 42% by 2019? 44% by 2020?

For Goal #3, the state plan presented the 2015 PARCC results, well below the proficiency rates than CSAP/TCAP had been telling us since the late 1990’s. (2016 scores were similar.)

PARCC - % of Colorado students at Benchmark
Grades 3-9	2015
English Language Arts	40%
Math	31%

Again, CDE’s Performance Plan looked ahead to ESSA: “The Department will establish appropriately ambitious and attainable CMASS PARCC goals through the ESSA state plan development, along with the 2016 assessment results.” But do they measure proficiency? No.

ESSA - Another set of goals

Here are Colorado’s GOALS found in the opening pages of the 150-page application (Section I – pp. 9-11) and detailed in APPENDIX A: Measurements of Interim Progress (pp. 146-147).

A. Academic Achievement. English Language Arts and Math

	Baseline Distribution of Current Year Data	Interim Target Year 2 on Baseline Distribution	Interim Target Year 4 on Baseline Distribution	Long-term Goal on Baseline Distribution
All students	50^th Percentile- ELA 50^th Percentile- Math	51^st Percentile ELA & Math	52^nd Percentile ELA & Math	53^rd Percentile ELA & Math

B. Graduation Rates

	Baseline Current Year Data	Interim Target Year 2	Interim Target Year 4	Long-term Goal
All students	82.5%	85.1%	87.7%	90.3%

C. English Language Proficiency

	Baseline Current Year Data	Interim Target Year 2	Interim Target Year 4	Long-term Goal
All students	12%	13%	14%	15%

What’s this, we ask? 50^th percentile in English and Math …? But as we have just seen, recent data tells us only 40% are proficient in English for grades 3-9, less than one-third in math.

In section IV of the ESSA application, under Accountability, we read: “To ensure that student privacy is maintained, Colorado has transitioned to the use of mean scale scores.” CDE has explained this shift to me—see Addendum C for a more complete explanation found in the full 150-page plan. It sounds well-intentioned: “This creates accountability for students that are struggling greatly and currently nowhere near meeting benchmark.” Maybe Colorado’s plan is “ambitious” enough and clear enough for the U.S. Department of Education to approve. (Interesting to see a few state plans already facing some tough questions[ii]. Will we be next?)

And yet the benefits of this transition to mean scale scores are not clear to me. I value the research and analysis of Bellwether Education Partners. Their criticism of Colorado’s ESSA plan (see excerpts in Addendum D) may seem harsh, but I believe this outside perspective deserves a wide reading by Colorado policymakers. “… the state’s plan does not set long-term goals for students’ early preparation, includes average scale scores rather than a proficiency measure, and does include any goals for college and career readiness.”

CDE will raise objections to several points made by Bellwether; moreover, I am sure anything written to fulfill federal regulations is bound to be a bit of a mess. But if it is fair to call “our long-term goals … disconnected from the state’s vision”—as seems true (measuring proficiency on one, using mean scale scores on another), surely we can do better.

My conclusion is that our ESSA plan brings us no closer to clear achievement goals we can understand and rally around and measure ourselves against, year after year.

State ESSA plans: Just BS, or reform's BFF? (PODCAST - July 5, 2017)

This podcast with Chad Aldeman of Bellwether Education Partners offers a fun and informative exchange with Michael J. Petrilli and Alyssa Schwenk of the Thomas B. Fordham Institute on the long-term goals required for ESSA. Listen to Petrilli mock the “utopian goals” in many state plans: “‘Every subgroup is going to magically be at the same level in 5 or 10 years.’” [NOTE: Colorado’s plan is a perfect example; see the final section in Addendum C and the stated goals for our ELL, Black, and Hispanic students in a few years.]

Petrilli continues: “… why even pretend that those goals matter? … they [state policymakers] can’t be honest; if everything goes right we will see incremental progress for kids and we will slowly see some slight narrowing of the achievement gaps – that is the best case scenario. And if you’re a policymaker you just can’t come right out and admit that.” https://edexcellence.net/commentary/podcasts/state-essa-plans-just-bs-or-reforms-bff?utm_source=Fordham+Updates&utm_campaign=2ddb0a4fe8- _term=0_d9e8246adf-2ddb0a4fe8-70657489&mc_cid=2ddb0a4fe8&mc_eid=c7b7dd90d4

Does it even matter whether we have clear, measurable goals for the state?

Do teachers and individual schools really care about all this? Probably not. We can only have an impact on the learning that takes place in our classroom and our building. It mattered to me as a member of a School Accountability Committee (SAC- 2002-04) to help our school establish meaningful CSAP goals. As an English teacher, I certainly shared with my classes the CSAP results from the previous year, and I set my own goals for each year’s group. Where school leaders and teachers feel accountable, goals matter. But beyond our building, 21-page state performance plans and 150-page documents (and long-winded newsletters) are easy to ignore.

But as one interested in education policy and the big picture, I believe we should establish clear and meaningful goals for Colorado. All of us who hope we are making a difference would value a reliable benchmark to see if there is progress. Critical, don’t you agree, if we are to decide whether to “stay the course”? Or if, instead, we have to acknowledge: even we are “inching forward,” it simply is not good enough.

***

Addendum A - Did we see progress during the 15 years using CSAP and TCAP?

Colorado CSAP/TCAP – 2005 - 2014 - % proficient and advanced

	2005	2006	2007	2008	2009	2010	2011	2012 TCAP	2013 TCAP	2014 TCAP	Change between 2005 and 2014	(GOAL for 2014 stated in Colorado’s RTTT application)
READING	66.3%	67.6%	67.2%	67.8%	68.3%	68.4%	67.9%	69.3%	69.5%	68.9%	+2.6% pts	85%

MATH	50.6%	52.4%	53.1%	53.2%	54.5%	54.9%	55.6%	55.8%	56.7%	56.4%	+5.8% pts	85%

http://www.cde.state.co.us/sites/default/files/documents/assessment/documents/newsreleases/2012/2012_tcap_tablsprelease.pdf (2005-12)

From 2008-2013 – “not much change” (2014)

“Since 2008, scores from the TCAP and its predecessor, CSAP, have gone up and down slightly, adding up to not much change in a state at the forefront of testing and considered a leader in education reform.” Eric Gorski, Yesenia Robles: http://www.denverpost.com/2014/08/14/colorado-students-show-slight-decline-in-state-tcap-results/

“…the lack of notable academic progress over 10 years…” (2014)

During a discussion at a Colorado State Board of Education meeting Thursday in Denver, some members said the lack of notable academic progress over 10 years and a large achievement gap between low-income students and more affluent students are disturbing.

"It's very, very troubling," said Denver board member Elaine Gantz Berman… "What we see is very bleak data here…. Sometimes we inch up, sometimes we inch down ... but we're not making the kind of gains that need to be made if we're going to be competitive….”

http://gazette.com/ugly-trend-in-colorado-tcap-scores-education-officials-seek-meaningful-change/article/1535553

4^th grade proficiency – increase by 10% points … but then – stagnation or a decline (2014)

A 2011 analysis of CSAP scores by I-News and Chalkbeat Colorado found that fourth-grade proficient and advanced levels in reading increased by 10 percentage points, from 55 to 65 percent, over the 15-year run of the CSAPs.

But that analysis also found that almost all the reading gains came in the first 10 years of testing, with most districts either stagnating or falling slightly since 2006.

http://www.coloradoindependent.com/148621/short-tcap-testing-era-ends-on-flat-note

Addendum B: From page 6 of AV#63 – Colorado’s RTTT Goal: How Credible….? (July 2010)

(NOT ABLE TO TRANSMIT THIS PAGE HERE - See hard copy of Another View #164.)

Addendum C - From Colorado ESSA Plan – pages 50-51

Align K–12 and postsecondary education goals

4.1 Accountability System.

Indicators. Describe the measure(s) included in each of the Academic Achievement, Academic Progress, Graduation Rate, Progress in Achieving English Language Proficiency, and School Quality or Student Success indicators and how those measures meet the requirements described in 34 C.F.R. § 200.14(a)-(b) and section 1111(c)(4)(B) of the ESEA.

The description for each indicator should include how it is valid, reliable, and comparable across all LEAs in the State, as described in 34 C.F.R. § 200.14(c)….

Indicator	Measure(s)	Description
i. Academic Achievement	Mean scale score	The mean scale score for each state-required content assessment in 3rd through 11th grades, in English Language Arts, mathematics, and science is included in the Academic Achievement indicator. This includes both traditional assessments and those aligned to the state’s alternate assessment standards for students with the most significant cognitive disabilities. To ensure that student privacy is maintained, Colorado has transitioned to the use of mean scale scores. This methodology has several other advantages over percent at benchmark (Polikoff, 2016) including that the performance of all students is reflected in the accountability metrics, not just those students who are close to the proficiency cut-scores. This creates accountability for students that are struggling greatly and currently nowhere near meeting benchmark, as well as for students who are above benchmark that can reach even higher levels. Mean scale scores provide similar performance inferences for school accountability as percent at benchmark. Finally, the percent of students scoring at benchmark will be reported publicly, as long as student data privacy is maintained. As state assessments are administered to meet federal requirements, they are subjected to the process of peer review by The U.S. Department of Education (USDE). This process ensures that assessments used for state summative reporting are aligned with the state’s academic content standards and are “valid, reliable, and consistent with relevant, nationally recognized professional and technical standards for the purposes for which they are used” (USDE, 2015). Colorado submitted the current battery of state assessments for peer review in 2016 and has received ratings of “substantially meets” for all assessments. Colorado will be working with the consortia and the USDE to provide the additional evidence requested. Since all public schools in Colorado annually administer the same required state assessments to all students, school-level results should be comparable statewide.
ii. Academic Progress	Median student growth percentile	The median student growth percentile for each of the CMAS English Language Arts and mathematics assessments in 4th through 9th grades will be included in the Academic Progress indicator. When an aligned system of high school assessments are fully implemented, Colorado plans to report median growth percentiles for high schools as well. Colorado has been using student growth percentiles calculated using a quantile regression model for many years. This normative metric describes a student’s observed progress in comparison to his or her academic peers. A number of research papers have been published exploring various facets of the student growth percentile model, its underlying calculations, aggregation possibilities, and uses for making school and district accountability inferences (Betebenner, 2009; Castellano, 2011; Dunn & Allen, 2009; Furgol, 2010). Additionally, the model was approved by USDE for use as part of the No Child Left Behind (NCLB) growth pilot in 2009, and has been adopted by numerous other states for various accountability and reporting purposes. When used and interpreted appropriately, growth percentiles are a valid measure of student learning and system improvement and demonstrate comparable technical qualities to other measures used for accountability reporting. Growth calculations are based on the required state assessments; as long as a large and sufficiently representative statewide sample of individuals is included, the student and aggregate results are comparable across all state systems (e.g. schools).

More details from Colorado’s ESSA plan and our long-term goals; an excerpt from the Appendix,

MEASUREMENTS OF INTERIM PROGRESS - page 147, figure 43:

A. Academic Achievement. English Language Arts and Math

Subgroups	Baseline Distribution of Current Year Data	Interim Target Year 2 on Baseline Distribution	Interim Target Year 4 on Baseline Distribution	Long-term Goal on Baseline Distribution
All students	50^th Percentile- ELA 50^th Percentile- Math	51^st Percentile ELA & Math	52^nd Percentile ELA & Math	53^rd Percentile ELA & Math
Economically disadvantaged students	18^th Percentile- ELA 19^th Percentile- Math	51^st Percentile ELA & Math	52^nd Percentile ELA & Math	53^rd Percentile ELA & Math
Children with disabilities	1^st Percentile- ELA 1^st Percentile- Math	51^st Percentile ELA & Math	52^nd Percentile ELA & Math	53^rd Percentile ELA & Math
English learners	16^th Percentile- ELA 19^th Percentile- Math	51^st Percentile ELA & Math	52^nd Percentile ELA & Math	53^rd Percentile ELA & Math
American Indian or Alaska Native	18^th Percentile- ELA 16^th Percentile- Math	51^st Percentile ELA & Math	52^nd Percentile ELA & Math	53^rd Percentile ELA & Math
Black	19^th Percentile- ELA 15^th Percentile- Math	51^st Percentile ELA & Math	52^nd Percentile ELA & Math	53^rd Percentile ELA & Math
Hispanic	21^st Percentile- ELA 20^th Percentile- Math	51^st Percentile ELA & Math	52^nd Percentile ELA & Math	53^rd Percentile ELA & Math

For Colorado’s ESSA State Plan, go to http://www.cde.state.co.us/fedprograms/essa

Addendum D

An Independent Review of ESSA State Plans, by Bellwether Education Partners* (6/27/2017)

(Bold mine)

From Executive Summary

This batch of ESSA plans marks another radical shift. As a country, we’re moving away from criterion-referenced accountability systems—where all schools are held to the same, predetermined criteria—to norm-referenced ones, where schools are compared to each other instead of to some external criteria. This change has given states the freedom to adopt more rigorous, and more honest, state standards and assessments, but it has also created a disconnect between the standards for students and the standards for schools. As one peer put it, states are now putting “the engine of a Mercedes in a Ford Taurus body.” That is, states have done the hard work of adopting more rigorous standards and more sophisticated assessments, but they’re only holding schools accountable for their place in relative ranking systems. Those systems ignore information on whether or not students are on track to succeed in college and careers. https://bellwethereducation.org/sites/default/files/Bellwether_ESSAReview_ExecSumm_Final_0.pdf

From Colorado –Project Overview (9 pages)

https://bellwethereducation.org/sites/default/files/Bellwether_ESSA_PlanReview_CO_Final.pdf

Under Weaknesses:

Colorado is proposing to shift to an entirely normative approach, where all indicators and the accountability system itself are based on relative performance, not a predefined standard. That approach may not be sufficiently clear to parents, educators, or other stakeholders, and it means the accountability system has no incentives aligned to the state’s professed goal of college and career readiness for all students. The state also lacks coherent goals for its schools, and the ones provided are disconnected from the state’s long-term vision. As a result, the system does not provide schools clear signals about how they need to improve, and it’s particularly problematic for those students who have been historically sidelined as a result of their race, class, and/or life circumstances. (page 1)

Under Plan Components: Each state’s plan has been rated on a scale of 1 (“This practice should be avoided by other states”) to 5 (“This could be a potential model for other states”).

Goals: Are the state’s vision, goals, and interim targets aligned, ambitious, and attainable? Why or why not?

Bellwether’s report gave Colorado its lowest score on this component, a 1 out of 5.

Colorado’s long-term goals are disconnected from the state’s vision. The vision lays out the priorities of all students demonstrating readiness for school, third-grade reading proficiency, meeting or exceeding standards throughout their schooling years, and graduating high school ready for college and careers. However, the state’s plan does not set long-term goals for students’ early preparation, includes average scale scores rather than a proficiency measure, and does not include any goals for college and career readiness. The enumerated long-term vision is also disconnected from the state’s system for classifying school performance.

Colorado has not set clear long-term goals or interim targets to reach its vision. Instead, the state has proposed a confusing percentile-based system that intends to raise the statewide performance from the 50th to the 53rd percentile. The goal to increase the average scale score from the 50th to the 53rd percentile statewide in six years is (1) difficult for parents, educators, and the public to understand; (2) does not set strong expectations for all schools and all groups of students to improve; and, (3) may not be ambitious improvement because the plan does not provide any information about the percentage of students meeting grade-level standards at that performance level. Each of these factors goes against best-practice research on goal setting.

As written, Colorado expects children with disabilities, who currently score at the 1st percentile statewide, to score at the 53rd percentile in six years. That would be an impressive gain, but Colorado also expects Asian students, who currently score at the 82nd percentile statewide, to regress backward to the same 53rd percentile within six years.

In contrast, Colorado has set graduation rate goals using objective data on past performance. The state set a goal of increasing its graduation rate to 90.3 percent within a six-year time frame, based on its analysis of what the state has achieved in recent years. But unlike the percentile approach, which is normative, graduation rate gains are based on actual, observed changes over time against a predefined threshold. The state should apply a similar approach to its achievement goals.

Lastly, Colorado’s plan lacks information and specificity about historical English-language proficiency performance, goals, and interim targets. (page 3)

Note also Bellwether’s low scores (2 out of 5) for Colorado’s plan for the sections on accountability indicators and academic progress.

*More on Bellwether Education Partners is at: https://bellwethereducation.org/.

[i] “Participation Facts: • Nationwide: 137,400 public school fourth- grade students in 7,230 schools participated. • Nationwide: 135,100 public school eighth-grade students in 5,670 schools participated. • Colorado: 2,200 public school fourth- grade students in 98 schools participated. • Colorado: 2,300 public school eighth-grade students in 94 schools participated.” (http://www.cde.state.co.us/assessment/naepsummaryreading4and8

[ii] Letter from U.S. Department of Education of Delaware on its ESSA plan: “Because the proposed long-term goals for academic achievement are not ambitious, DDOE must revise its plan ….” https://www2.ed.gov/admins/lead/account/stateplan17/deprelimdetermltr.pdf

ALSO -

Every Student Succeeds Act Consolidated State Plan - Frequently Asked Questions June 16, 2017 – from U.S. DOE -

5. If the statute does not define certain terms, such as “ambitious” long-term goals, “substantial” weight of indicators, or “much greater” weight of certain indicators over others, who determines the meaning of those terms? In cases where the statute does not define a specific term, a State has significant discretion to determine how it will define that term. In accordance with the Secretary’s responsibility to review State plans, the Secretary is obligated to make a determination as to whether a State’s proposed definition, on its face, is reasonable. https://www2.ed.gov/policy/elsec/leg/essa/essastateplansfaqs.pdf

Another View

Wednesday, July 19, 2017

AV#164 - K-12 public education in Colorado - Are we making progress? How to measure?

C. English Language Proficiency

MEASUREMENTS OF INTERIM PROGRESS - page 147, figure 43:

A. Academic Achievement. English Language Arts and Math