#3 in a series on Accountability
“So how do we move towards assessments that are taking advantage of some of the technology that we have, … towards assessments that might be more adaptive?” Former U. S. Secretary of Education Dr. John B. King (See Addendum A)
If AV #222 convinced
you that we need to find other ways to assess the academic performance and
growth in many of our high schools, here are two such efforts in Colorado. My
goal in #223 is merely to introduce you to these options, with links that allow
you to explore them—and similar endeavors in other states. All of us who advocate
for school accountability need not succumb to the idea that one-size fits all.
Student-Centered
Accountability Program (SCAP- 15 districts)
Alternative
Education Campus (100 schools)
Neither approach opts out of PSAT and SAT testing, and yet both have found additional ways for schools to report on the achievement of their students.
|
Support from the
Colorado legislature for new options In passing SB 19-204, legislators invited districts to pilot their own local accountability systems that “supplement the state system.” Several exciting efforts are underway. See examples from Falcon 49 and Jefferson County in this video presentation from the Colorado Education Initiative, https://vimeo.com/481467491/b364afc26e. CEI provides links for more on SB 19-204 and efforts beyond Colorado: see Addendum C. |
I see nothing radical in expanding the options in how high
schools assess their students and provide valid results as part of a more flexible
accountability system. In SB 19-204 (see box), the state has welcomed new
ideas. Moreover, the state already expanded the “menu” of our Graduation
Requirements. Minimum SAT scores are just one of a number of ways that
students can demonstrate they have met a certain standard in order to graduate
(Addendum B). The minimum—470 in English, 500 in Math—is out of reach
for perhaps one-third of our juniors and seniors. Our new “menu” offers other
options.
It is much the same with high school assessments. We need more options—not to lower the bar, but to use tests that teachers and students will find more meaningful than they do the PSAT and SAT.
1.
Student-Centered Accountability Program (S-CAP) – rural schools –
n<16 and pages with no data
“A system of accountability proposed and designed by a group of Colorado rural school districts. At the heart of Student-Centered Accountability is a focus on the success of well-rounded students using a system for continuous improvement. To accomplish this, the districts use multiple measures of student success to expand results beyond a single state test score.” https://scapbvschools.weebly.com/
Rebecca Holmes, President and CEO of Colorado Education Initiative, and Lisa Yates, superintendent of the Buena Vista School District, wrote a guest commentary in The Denver Post that spoke specifically to S-CAP. I present their words here as they apply to both efforts featured in AV #223.
It’s time to change Colorado’s school
accountability system
Rural areas are
making progress that state should explore (Bold mine)
“… the
experience of local educators in recent years has made it clear it is time for
another significant shift. Our accountability system allows for comparisons
across schools and districts at the state level, but it does not provide the
data in an actionable way that allows for timely intervention or information
about leading indicators of improvement. Yes, our current accountability
system has exposed a deep achievement gap, but it does not provide much help in
understanding how to close it.
“It’s time to
take the example that local districts have set — and encourage other educators
as well as the state to once again lead the way in rethinking accountability.
We need to bring our district and school leaders to the collective table,
soliciting their input on how to update the accountability system so that
districts and schools feel empowered by a model that educators and families
believe in.”[i]
Holmes and Yates
were referring to the example set by the (now) 15 districts involved in S-CAP. They
have created a “Comprehensive Student Success Dashboard,” which provides
measures in two categories:
“The purpose of the CSS Dashboard is to provide our community with a more comprehensive understanding of student success, to enhance transparency of student performance, to provide timely results of student growth and achievement, and to broaden the definition of student success by expanding measures beyond annual state testing results.”
|
Examples from three of S-CAP’s high schools:
·
Buena Vista and Holyoke - Achievement and Growth - Standardized Assessments
“The following NWEA MAP standardized assessments provide a thorough picture of student learning: how much they've grown, what they're ready to learn, and the specific skill gaps they need to master.”
Buena Vista - https://scapbvschools.weebly.com/standardizedassessments.html
Holyoke - https://holyokescap.weebly.com/standardizedassessments.html
·
Kit Carson – reporting includes data on 9th
and 10th graders
Fall to Spring Percentage Growth - MAPS Winter to Spring RIT Scores - https://kitcarsonscap.weebly.com/uploads/8/5/3/7/85377236/jh-hs_fall_to_spring_maps_and_rit_2018.pdf
|
Rural School
Districts, Rural High Schools 2019 SPF for at least 50
districts has no SAT data Chalkbeat
Colorado’s review of the 2019 SAT
scores included this observation. “The
state withheld test score data for about a quarter of all high schools
because of the small number of students who took the tests there. The
practice is meant to protect student privacy, but several advocacy and
research organizations have criticized the policy, saying it is too broad and
hides important information from the public.”[iii] About a quarter of all high schools? Maybe too high a guess, but it surely totals more than a quarter of all Colorado school districts. I count at least 50 districts (out of 186) where the number (n) of students taking the SAT was less than 16, and therefore we see no data on the Postsecondary and Workforce Readiness page, just this: CO SAT – R & W – All students n<16 CO SAT – Math – All students n<16 (In these districts, the page titled Academic Achievement, with PSAT results, is often equally empty.) For 32 of the 33 Colorado school districts with fewer than 175 students, their SAT scores were Redacted due to insufficient numbers. Is it any wonder that S-CAP is looking at ways to include other assessments, where a rural school does not have to wait until late in the summer before seeing that its School Performance Framework reports … nothing … on its students’ PSAT and SAT scores? How useful is that? Holmes and Yates ask for an updated accountability
system, “a model that educators and families believe in.” Hard to “believe in”
an SPF that reveals little to nothing to 50 districts on the academic
achievement, growth, or postsecondary and workforce readiness of their high
school students. |
2.
Alternative Education Campuses - AEC
schools serve “high-risk” students – options expected
Options include a range of other assessments. Here is one example. From the AEC School Performance Framework in 2019 for Pathways Future Center in Adams 12 Five Star Schools.[vi]
*Percent of
students scoring at or increasing at least one grade level
POSTSECONDARY & WORKFORCE READINESS
|
|
|
Subject |
Count |
Result |
% Pts Earned |
Rating |
|
Required State Measures |
Co SAT |
R & W |
24 |
447.5 |
75% |
Meets |
|
|
|
Math |
24 |
428.8 |
100% |
Exceeds |
|
Optional Measures |
Graduation |
Graduation |
940 |
71.8% |
100% |
Exceeds |
|
|
WORKKEYS Certificate |
Certificate Earned |
547 |
77.5%* |
75% |
Meets |
*Percent of students who earned at least a Bronze
rating on the National Career Readiness Certificate
Other AEC
schools, like New Legacy Charter School and New America School–Lowry,
also use the NWEA MAP. (See more from their authorizer, the Colorado Charter
School Institute, in Addendum D.) In addition, under Optional
Measures on Postsecondary & Workforce Readiness, we see:
New Legacy - Credit
Course Completion - Percent of credits earned out of credits attempted by
course
-
Postsecondary Acceptance – Percent of students who were accepted to an
institution of higher learning after completion
New America
School - Post-Completion Success
Rate – Percent of students with positive educational or workforce outcomes
following completion
Computer
adaptive tests
S-CAP and AEC’s options both welcome the use of NWEA MAP assessments. Why is that important? Because they are computer adaptive tests (more in Addendum D and E). They adjust to the ability of the student taking the test, based on how the first few questions are answered. They do what good teachers are encouraged to do, they differentiate, while still providing useful information as to how well a student is meeting our standards in reading and math. (Understandably, such quick results are not possible with writing assessments.) The test itself is meeting students where they are, rather than insisting they demonstrate their “college readiness”—as the PSAT and SAT are designed to do.
Less frustrating
for students. More useful to teachers. A better option? Worth a closer study? I
hope so.
Addendum A (All bold mine)
Dr. John B. King,
former U.S. Secretary of Education
Dr. King was asked about assessment in the time of Covid. His answer was a remarkable synthesis, touching on what parents need to know, on civil rights, and on using technology to redefine assessments. Wise words.
“… I'd start with the values, what are the values
behind why we have annual assessments. One is parents and teachers need good
information about how students are doing. We've done some polling recently … we
hear very consistently from parents during this Covid period especially, is how
desperately they want more information about how their kids are doing. They
want to know whether or not their kids are falling behind. There's real fear
about kids falling behind.
“That's one of the values at stake.
|
The focus in section 3 in last week’s Another View: Equity –
Students on Free or Reduced Lunch and English Learners - PSAT/SAT Results -
pp. 8-10. |
“... And one of the things that is very important to the civil rights community is the ability to take a hard look at where there are gaps in opportunity based on race, based on income, based on English learners status, based on disability. So those are the values that ought to animate how we think about this question of what to do about assessments. What I hope will happen is we have to see where we are with the pandemic. But I hope states will stand up the capacity to provide assessments, whether it's online or in person. And we know that that is possible to do at scale so that we have information this year on how students are doing.
“But longer-term, I think there is a lot of work to do on what I would characterize as assessment innovation. So how do we move towards assessments that are taking advantage of some of the technology that we have, having moved towards assessments that might be more adaptive? How do we move towards reading assessments that might build in more content knowledge? Louisiana is piloting reading assessments that tie reading with social studies content. So we get a sense of whether or not students are building the knowledge base that they need to be successful readers. How do we build in more performance-based opportunities? … So I do think we're going to see more of that kind of assessment innovation over time.
*https://drive.google.com/file/d/1zfsabEJXcsdXfcp3c9-kE-vMP9K9OJz9/view
Addendum B
Chalkbeat
Colorado, May 29, 2020.[vii]
“In Colorado, graduation requirements are set
by districts. But the state approved requirements that students, starting with
next year’s seniors, show a minimum proficiency in English and math. The state
did not tell districts how students must show that proficiency, but
instead offered
a menu of ways districts could offer for students to prove that
knowledge.”
Addendum C
Other resources and three states: Massachusetts, New York,
and New Hampshire
Senate Bill 19-204 - From the Colorado Education Initiative video - https://vimeo.com/481467491/b364afc26
·
Creates local accountability
system grant program
·
Provides $ to support
development of local accountability systems that supplement state system
·
Provides incentives to develop a
local system
·
[CDE] is required to report
“learnings” from the pilots to the legislature on an annual basis
For more information on SB
19-204 - http://www.cde.state.co.us/localaccountabilitysystemgrant.
For CEI Series on Accountability
– https://www.coloradoeducationinitiative.org/to-whom-are-schools-most-accountable/
For reports and more examples of
work on options for assessments beyond the PSAT and SAT, see:
·
RCA
CUNY Accessing College Readiness
- The
Promise of Performance Assessments
- Including
Performance Assessments in Accountability Systems
- Ready
for the Future
Massachusetts
·
Massachusetts Consortium for Innovative
Education Assessment (MCIEA)
“Restoring the broader purpose of education. By
measuring school quality in a fair and comprehensive way, MCIEA seeks to
reaffirm the full mission of public education. MCIEA believes it is possible to
track school quality without relying on a narrow set of indicators and in a way
that reflects the unique character of each school community.”
https://www.mciea.org/school-quality-measures.html
“The New Accountability” - https://www.mciea.org/uploads/1/2/0/7/120788330/mciea_2019_forum_pamphlet.pdf
I am most grateful to the Colorado Education
Initiative for passing along these additional resources.
**
New York and New Hampshire
This guest commentary in The Denver Post (August 16, 2019) pointed to similar “experiments” in New York and New Hampshire. By Andrea Gabor for Bloomberg Opinion.
It might be easy to say good riddance [to all the tests imposed on public education of late], but schools still need ways to measure student progress. The accountability movement that pushed testing was a response to a genuine need to improve K-12 education…
So
schools need to find new ways to show accountability advocates that test
retrenchment won’t weaken standards, and this presents an opportunity to
develop more robust assessments and better education.
The country’s best under-the-radar experiments are a useful guide. Chief among these is the New York State Performance Standards Consortium…
In 2015, New Hampshire won a waiver under a federal pilot program that opened the door to alternative assessment programs, and is introducing performance-based projects like New York’s that are designed almost entirely by teachers.
Addendum D
By
Jean Fleming, NWEA blog, March 17, 2016
|
Ryan Marks, Director of
Evaluation and Assessment at Colorado’s Charter School Institute, spoke with
me about several benefits of this computer adaptive test. “The NWEA is more relevant”
for these AEC students. Given three times a year, fall, winter and spring,
“teachers and students can see their progress.” Yes, “it feels a bit easier”
(than the SAT). By adjusting the question (based on their answers), “it makes
the test accessible.” |
“Adaptive tests—like our Measures of Academic Progress® (MAP®) interim assessments—can pinpoint student
growth and instructional needs accurately in a relatively short amount of time.
MAP assessments take about an hour, and give students and educators information
they can immediately use to move learning forward.
“No single assessment can do it all—nor should it. Using multiple measures allows educators to cross-check their data and answer different educational questions with the appropriate tools. But when it comes to driving individual learning, especially for underserved populations, formative and interim assessments have a critical role to play in providing the information educators need to close achievement gaps. To understand where all students are on their learning path, an adaptive assessment can be an invaluable tool, provided it meets certain criteria: measuring growth regardless of grade and gathering data efficiently.”
Addendum E
“How the ACT
and SAT exams are built to fail students - We have the technology to do better”
By Dr. Iain Harlow, Podium, March 16, 2019
“Despite the growing
movement against using ACT and SAT exams as the be-all-end-all benchmark for
assessing college readiness, more students than ever are taking them – roughly
four million in 2018. The results haven’t been promising. More
than half of SAT takers still aren’t considered ready for
college-level courses, while recent ACT scores actually showed a drop in overall college
readiness.
“Why are the results
so poor? Are the underperforming test takers simply not ready for college?
The answer to these questions is the very reason many in the field of cognitive
science believe these exams shouldn’t matter in the first place –
moment-in-time evaluations are fraught with problems and don’t provide an
accurate view of real knowledge or potential. We have the technology to do
better.
“Today, the ACT and SAT matter – a lot. But
as learner tools and data continue to improve the learning experience, they shouldn’t.
The real test won’t be the ability to stay cool, complete test forms, and
outsmart exam day, it will be objectively tracked, long-term knowledge and
understanding.
“As we move to online, on-demand
curriculum and assessment, the data on student performance, cognition, and
ability to learn will only increase and standalone tests will matter less. That
would be a perfect score.”
https://thenextweb.com/podium/2019/03/16/how-the-act-and-sat-exams-are-built-to-fail-students/
Addendum F
A recent essay in The Economist on
human rights around the world examined the various laws and strategies to bring
to account those who abuse human rights. It featured the principal of universal
jurisdiction, the International Criminal Court (ICC), the United Nation’s Human
Rights Council (HRC), and UN backed-international courts and tribunals. Such an
extensive list may seem chaotic, too diverse to enforce human rights across the
globe. The essay concluded, though, that it helps to have more than one
approach.
"While the ICC has struggled, other 'hybrid' courts have been
more effective, embracing a medley of ad hoc judicial systems operating under a
mix of home-grown and foreign judges. 'The ICC will never be a solution for all
human-rights violations,' says Serge Brammertz, a long-serving chief prosecutor
for the Yugoslav tribunal. 'I believe in an à la carte system: the ICC has an important but not the only role.'" "No time to give up," The
Economist, Jan. 2, 2021. (emphasis mine)
Endnotes
[i] The Denver Post, Dec. 18, 2018, https://www.denverpost.com/2018/12/18/school-accountability-requires-ability-react-quickly/
[ii]
The
Comprehensive Student Success Dashboard (CSS) provides measures in two
categories: Academics and Learning Dispositions.
The purpose of the CSS Dashboard is to provide our community with a more
comprehensive understanding of student success, to enhance transparency of
student performance, to provide timely results of student growth and
achievement, and to broaden the definition of student success by expanding
measures beyond annual state testing results.
[iii]
https://co.chalkbeat.org/2019/8/15/21108632/sat-scores-are-slightly-down-in-colorado-find-your-school-s-2019-results
[vii]
“Grad
requirements, seat time and other statewide school issues to watch in
Colorado,” Chalkbeat Colorado, by Yesenia
Robles, May 29,
2020.
https://co.chalkbeat.org/2020/5/29/21274015/five-education-issues-to-watch-for-next-school-year-colorado
