Tuesday, January 19, 2021

AV #223 - Accountability: Besides PSAT/SAT, how else can high schools measure their performance?

#3 in a series on Accountability

“So how do we move towards assessments that are taking advantage of some of the technology that we have, … towards assessments that might be more adaptive?”       Former U. S. Secretary of Education Dr. John B. King (See Addendum A)


If AV #222 convinced you that we need to find other ways to assess the academic performance and growth in many of our high schools, here are two such efforts in Colorado. My goal in #223 is merely to introduce you to these options, with links that allow you to explore them—and similar endeavors in other states. All of us who advocate for school accountability need not succumb to the idea that one-size fits all.

Student-Centered Accountability Program (SCAP- 15 districts)

Alternative Education Campus (100 schools)

Neither approach opts out of PSAT and SAT testing, and yet both have found additional ways for schools to report on the achievement of their students.


Support from the Colorado legislature for new options

In passing SB 19-204, legislators invited districts to pilot their own local accountability systems that “supplement the state system.” Several exciting efforts are underway.  See examples from Falcon 49 and Jefferson County in this video presentation from the Colorado Education Initiative, https://vimeo.com/481467491/b364afc26e.     CEI provides links for more on SB 19-204 and efforts beyond Colorado: see Addendum C.

I see nothing radical in expanding the options in how high schools assess their students and provide valid results as part of a more flexible accountability system. In SB 19-204 (see box), the state has welcomed new ideas. Moreover, the state already expanded the “menu” of our Graduation Requirements. Minimum SAT scores are just one of a number of ways that students can demonstrate they have met a certain standard in order to graduate (Addendum B). The minimum—470 in English, 500 in Math—is out of reach for perhaps one-third of our juniors and seniors. Our new “menu” offers other options.


It is much the same with high school assessments. We need more options—not to lower the bar, but to use tests that teachers and students will find more meaningful than they do the PSAT and SAT.


1.      Student-Centered Accountability Program (S-CAP) – rural schools – n<16 and pages with no data

“A system of accountability proposed and designed by a group of Colorado rural school districts. At the heart of Student-Centered Accountability is a focus on the success of well-rounded students using a system for continuous improvement. To accomplish this, the districts use multiple measures of student success to expand results beyond a single state test score.” https://scapbvschools.weebly.com/ 

Rebecca Holmes, President and CEO of Colorado Education Initiative, and Lisa Yates, superintendent of the Buena Vista School District, wrote a guest commentary in The Denver Post that spoke specifically to S-CAP. I present their words here as they apply to both efforts featured in AV #223. 

It’s time to change Colorado’s school accountability system

  Rural areas are making progress that state should explore                 (Bold mine)

“… the experience of local educators in recent years has made it clear it is time for another significant shift. Our accountability system allows for comparisons across schools and districts at the state level, but it does not provide the data in an actionable way that allows for timely intervention or information about leading indicators of improvement. Yes, our current accountability system has exposed a deep achievement gap, but it does not provide much help in understanding how to close it.           

“It’s time to take the example that local districts have set — and encourage other educators as well as the state to once again lead the way in rethinking accountability. We need to bring our district and school leaders to the collective table, soliciting their input on how to update the accountability system so that districts and schools feel empowered by a model that educators and families believe in.”[i]


Holmes and Yates were referring to the example set by the (now) 15 districts involved in S-CAP. They have created a “Comprehensive Student Success Dashboard,” which provides measures in two categories:

         Academics and Learning Dispositions.[ii]    

​“The purpose of the CSS Dashboard is to provide our community with a more comprehensive understanding of student success, to enhance transparency of student performance, to provide timely results of student growth and achievement, and to broaden the definition of student success by expanding measures beyond annual state testing results.”   


                                       À la carte                                                         Too many choices? Consider why some argue—with a far more complex issue, human rights across the globe—it seems best.  See Addendum F

Examples from three of S-CAP’s high schools: 

·        Buena Vista and Holyoke - Achievement and Growth - Standardized Assessments

“The following NWEA MAP standardized assessments provide a thorough picture of student learning: how much they've grown, what they're ready to learn, and the specific skill gaps they need to master.” 

Buena Vista - https://scapbvschools.weebly.com/standardizedassessments.html  

Holyoke - https://holyokescap.weebly.com/standardizedassessments.html  

·        Kit Carson – reporting includes data on 9th and 10th graders

Fall to Spring Percentage Growth - MAPS                                                                                                                                               Winter to Spring RIT Scores - https://kitcarsonscap.weebly.com/uploads/8/5/3/7/85377236/jh-hs_fall_to_spring_maps_and_rit_2018.pdf 


Rural School Districts, Rural High Schools

2019 SPF for at least 50 districts has no SAT data 

   Chalkbeat Colorado’s review of the 2019 SAT scores included this observation.

          “The state withheld test score data for about a quarter of all high schools because of the small number of students who took the tests there. The practice is meant to protect student privacy, but several advocacy and research organizations have criticized the policy, saying it is too broad and hides important information from the public.”[iii]

   About a quarter of all high schoolsMaybe too high a guess, but it surely totals more than a quarter of all Colorado school districts. I count at least 50 districts (out of 186) where the number (n) of students taking the SAT was less than 16, and therefore we see no data on the Postsecondary and Workforce Readiness page, just this:

CO SAT – R & W – All students   n<16                                                                                    CO SAT – Math   – All students   n<16

(In these districts, the page titled Academic Achievement, with PSAT results, is often equally empty.) For 32 of the 33 Colorado school districts with fewer than 175 students, their SAT scores were Redacted due to insufficient numbers. 

   Is it any wonder that S-CAP is looking at ways to include other assessments, where a rural school does not have to wait until late in the summer before seeing that its School Performance Framework reports … nothing … on its students’ PSAT and SAT scores? How useful is that? 

   Holmes and Yates ask for an updated accountability system, “a model that educators and families believe in.” Hard to “believe in” an SPF that reveals little to nothing to 50 districts on the academic achievement, growth, or postsecondary and workforce readiness of their high school students.


2.       Alternative Education Campuses - AEC schools serve “high-risk” students – options expected

 AV #222 showed the disastrously low SAT scores common to many AEC schools. It is good that the state has created much-needed flexibility for the AECs in what they report for the School Performance Framework. “AECs are permitted to select optional measures for accountability and improvement planning in addition to the state-required measures” (CDE’s AEC Overview[iv]). In a 2019 update, “State Required Measures and Policy for the AEC SPF,” we see that CDE has “encouraged districts to submit supplemental measures for their AECs in each indicator if statewide assessments are not available and sufficient data is potentially unavailable.”[v] The encouragement has teeth: the AEC would be “lowered one school plan type on the final SPF if it chose not to submit optional measures.” (emphasis mine)

Options include a range of other assessments. Here is one example. From the AEC School Performance Framework in 2019 for Pathways Future Center in Adams 12 Five Star Schools.[vi]







% Pts Earned


Required State Measures


R & W












Optional Measures

NWEA MAP Achievement













*Percent of students scoring at or increasing at least one grade level







% Pts Earned


Required State Measures


R & W












Optional Measures








WORKKEYS Certificate

Certificate Earned





*Percent of students who earned at least a Bronze rating on the National Career Readiness Certificate


Other AEC schools, like New Legacy Charter School and New America School–Lowry, also use the NWEA MAP. (See more from their authorizer, the Colorado Charter School Institute, in Addendum D.) In addition, under Optional Measures on Postsecondary & Workforce Readiness, we see:

New Legacy   - Credit Course Completion - Percent of credits earned out of credits attempted by course

 - Postsecondary Acceptance – Percent of students who were accepted to an institution of higher learning after completion

New America School -  Post-Completion Success Rate – Percent of students with positive educational or workforce outcomes following completion 


Computer adaptive tests 

S-CAP and AEC’s options both welcome the use of NWEA MAP assessments. Why is that important? Because they are computer adaptive tests (more in Addendum D and E). They adjust to the ability of the student taking the test, based on how the first few questions are answered. They do what good teachers are encouraged to do, they differentiate, while still providing useful information as to how well a student is meeting our standards in reading and math. (Understandably, such quick results are not possible with writing assessments.) The test itself is meeting students where they are, rather than insisting they demonstrate their “college readiness”—as the PSAT and SAT are designed to do. 

Less frustrating for students. More useful to teachers. A better option? Worth a closer study? I hope so.


                                                             Addendum A                                       (All bold mine)

 Dr. John B. King, former U.S. Secretary of Education

 From The Leader’s Table* - A Leadership Salon with Dr. John B. King, CEO of The Education Trust, Leadership for Educational Equity, Nov. 18, 2020 

Dr. King was asked about assessment in the time of Covid. His answer was a remarkable synthesis, touching on what parents need to know, on civil rights, and on using technology to redefine assessments. Wise words. 

“… I'd start with the values, what are the values behind why we have annual assessments. One is parents and teachers need good information about how students are doing. We've done some polling recently … we hear very consistently from parents during this Covid period especially, is how desperately they want more information about how their kids are doing. They want to know whether or not their kids are falling behind. There's real fear about kids falling behind.

“That's one of the values at stake. 

The focus in section 3 in last week’s Another View: Equity – Students on Free or Reduced Lunch and English Learners - PSAT/SAT Results - pp. 8-10.

“... And one of the things that is very important to the civil rights community is the ability to take a hard look at where there are gaps in opportunity based on race, based on income, based on English learners status, based on disability. So those are the values that ought to animate how we think about this question of what to do about assessments. What I hope will happen is we have to see where we are with the pandemic. But I hope states will stand up the capacity to provide assessments, whether it's online or in person. And we know that that is possible to do at scale so that we have information this year on how students are doing.     

“But longer-term, I think there is a lot of work to do on what I would characterize as assessment innovation. So how do we move towards assessments that are taking advantage of some of the technology that we have, having moved towards assessments that might be more adaptive? How do we move towards reading assessments that might build in more content knowledge? Louisiana is piloting reading assessments that tie reading with social studies content. So we get a sense of whether or not students are building the knowledge base that they need to be successful readers. How do we build in more performance-based opportunities? … So I do think we're going to see more of that kind of assessment innovation over time.

 “And hopefully, that moves us past some of the binary debates we've had about assessment that I think are a bit anachronistic. If we can leverage the technology to redefine assessments, assessments could even be built into the curriculum more so we might have students who are participating in expeditionary learning, for example, a really powerful English language arts curriculum that they do. Could there be assessments that are built into that and could that help give us information about how students are doing?”  



Addendum B

 “A menu of ways” to meet graduation requirements

Chalkbeat Colorado, May 29, 2020.[vii] 

“In Colorado, graduation requirements are set by districts. But the state approved requirements that students, starting with next year’s seniors, show a minimum proficiency in English and math. The state did not tell districts how students must show that proficiency, but instead offered a menu of ways districts could offer for students to prove that knowledge.”


Addendum C 

Other resources and three states: Massachusetts, New York, and New Hampshire 

Senate Bill 19-204 - From the Colorado Education Initiative video - https://vimeo.com/481467491/b364afc26

·        Creates local accountability system grant program

·        Provides $ to support development of local accountability systems that supplement state system

·        Provides incentives to develop a local system

·        [CDE] is required to report “learnings” from the pilots to the legislature on an annual basis

For more information on SB 19-204 - http://www.cde.state.co.us/localaccountabilitysystemgrant.

For CEI Series on Accountability – https://www.coloradoeducationinitiative.org/to-whom-are-schools-most-accountable/

For reports and more examples of work on options for assessments beyond the PSAT and SAT, see:

·        RCA CUNY Accessing College Readiness


·        Massachusetts Consortium for Innovative Education Assessment (MCIEA)

“Restoring the broader purpose of education. By measuring school quality in a fair and comprehensive way, MCIEA seeks to reaffirm the full mission of public education. MCIEA believes it is possible to track school quality without relying on a narrow set of indicators and in a way that reflects the unique character of each school community.”  


“The New Accountability” -  https://www.mciea.org/uploads/1/2/0/7/120788330/mciea_2019_forum_pamphlet.pdf


   I am most grateful to the Colorado Education Initiative for passing along these additional resources.


New York and New Hampshire

This guest commentary in The Denver Post (August 16, 2019) pointed to similar “experiments” in New York and New Hampshire. By Andrea Gabor for Bloomberg Opinion.    

   It might be easy to say good riddance [to all the tests imposed on public education of late], but schools still need ways to measure student progress. The accountability movement that pushed testing was a response to a genuine need to improve K-12 education… 

   So schools need to find new ways to show accountability advocates that test retrenchment won’t weaken standards, and this presents an opportunity to develop more robust assessments and better education.

   The country’s best under-the-radar experiments are a useful guide. Chief among these is the New York State Performance Standards Consortium…

   In 2015, New Hampshire won a waiver under a federal pilot program that opened the door to alternative assessment programs, and is introducing performance-based projects like New York’s that are designed almost entirely by teachers.



Addendum D

 “Adaptive Assessments – Meeting Students Where They Are in Their Learning”[viii]

By Jean Fleming, NWEA blog, March 17, 2016


Ryan Marks, Director of Evaluation and Assessment at Colorado’s Charter School Institute, spoke with me about several benefits of this computer adaptive test. “The NWEA is more relevant” for these AEC students. Given three times a year, fall, winter and spring, “teachers and students can see their progress.” Yes, “it feels a bit easier” (than the SAT). By adjusting the question (based on their answers), “it makes the test accessible.”

   “Adaptive tests—like our Measures of Academic Progress® (MAP®) interim assessments—can pinpoint student growth and instructional needs accurately in a relatively short amount of time. MAP assessments take about an hour, and give students and educators information they can immediately use to move learning forward.     

   “No single assessment can do it all—nor should it. Using multiple measures allows educators to cross-check their data and answer different educational questions with the appropriate tools. But when it comes to driving individual learning, especially for underserved populations, formative and interim assessments have a critical role to play in providing the information educators need to close achievement gaps. To understand where all students are on their learning path, an adaptive assessment can be an invaluable tool, provided it meets certain criteria: measuring growth regardless of grade and gathering data efficiently.”



Addendum E 

 “How the ACT and SAT exams are built to fail students - We have the technology to do better”

By Dr. Iain Harlow, Podium, March 16, 2019 

   “Despite the growing movement against using ACT and SAT exams as the be-all-end-all benchmark for assessing college readiness, more students than ever are taking them – roughly four million in 2018. The results haven’t been promising. More than half of SAT takers still aren’t considered ready for college-level courses, while recent ACT scores actually showed a drop in overall college readiness.

   “Why are the results so poor? Are the underperforming test takers simply not ready for college? The answer to these questions is the very reason many in the field of cognitive science believe these exams shouldn’t matter in the first place – moment-in-time evaluations are fraught with problems and don’t provide an accurate view of real knowledge or potential. We have the technology to do better.

   “Today, the ACT and SAT matter – a lot. But as learner tools and data continue to improve the learning experience, they shouldn’t. The real test won’t be the ability to stay cool, complete test forms, and outsmart exam day, it will be objectively tracked, long-term knowledge and understanding.

   “As we move to online, on-demand curriculum and assessment, the data on student performance, cognition, and ability to learn will only increase and standalone tests will matter less. That would be a perfect score.”



Addendum F

 À la carte – Perhaps the SAT "has an important but not the only role" 

     A recent essay in The Economist on human rights around the world examined the various laws and strategies to bring to account those who abuse human rights. It featured the principal of universal jurisdiction, the International Criminal Court (ICC), the United Nation’s Human Rights Council (HRC), and UN backed-international courts and tribunals. Such an extensive list may seem chaotic, too diverse to enforce human rights across the globe. The essay concluded, though, that it helps to have more than one approach.

"While the ICC has struggled, other 'hybrid' courts have been more effective, embracing a medley of ad hoc judicial systems operating under a mix of home-grown and foreign judges. 'The ICC will never be a solution for all human-rights violations,' says Serge Brammertz, a long-serving chief prosecutor for the Yugoslav tribunal. 'I believe in an à la carte system: the ICC has an important but not the only role.'" "No time to give up," The Economist, Jan. 2, 2021. (emphasis mine)



[ii] The Comprehensive Student Success Dashboard (CSS) provides measures in two categories: Academics and Learning Dispositions.  
​The purpose of the CSS Dashboard is to provide our community with a more comprehensive understanding of student success, to enhance transparency of student performance, to provide timely results of student growth and achievement, and to broaden the definition of student success by expanding measures beyond annual state testing results.  

[vii]Grad requirements, seat time and other statewide school issues to watch in Colorado,” Chalkbeat Colorado, bYesenia Robles, May 29, 2020.  https://co.chalkbeat.org/2020/5/29/21274015/five-education-issues-to-watch-for-next-school-year-colorado