Monday, April 15, 2019

AV #193 - Assessing reading and writing: different skills, different results - Part 2 - 2002-2018


 Part 2 – 2002 - 2018

ELA scores do not provide schools, teachers, parents, or students critical information


In last week’s newsletter (AV#192 - Part 1–1993-2001 - https://anotherviewphj.blogspot.com/) I recounted Colorado’s implementation of and commitment to our reading and writing standards, and to assessments that gave us useful information on where students performed as both readers and as writers. This week you will see how we maintained this focus for more than a decade, up until 2015. Today annual reports on the Colorado Measures of Academic Success (CMAS: Mathematics, English Language Arts (ELA), Science and Social Studies Data and Results[i]) no longer provide a distinction between a student’s proficiency in reading versus writing. We see only one ELA score (e.g. Grade 3 - 40.4%). Although state law changed to allow this (more later; see Addendum A), I wonder if this is what legislators intended (see “elimination”).  Or if this is what teachers and parents want—and need.

2018 CMAS Achievement Results[ii]
Subject
Grade
% Met or Exceeded Expectations
Mathematics
3
39%
Social Studies
4
22.4%
Science
5
35.5%
Reading
all
Not available
Writing
all
Not available
Today the state reports on the percentage of students in various grades Meeting or Exceeding Expectations in addressing the state standards in math, science, and social studies—but not on reading. And not on writing. Do you see the problem now?

Or try this: The state’s 2018 ELA score tells us 40.4% of third graders Met or Exceeded Expectations. Assume today’s gap in reading and writing for third graders matches what we saw in 2014 on the final TCAP score: 72% proficient in reading, 51% in writing. Let’s create a 2018 and 2014 comparison. Reduce each TCAP score by a third: 48% in reading, 34% in writing. Together, that might give us an ELA score of 41%. (I know, nothing scientific—but what if?) What if we saw a gap of 14% points like that from the CMAS test? How would it impact our debate on the READ Act, literacy instruction, and much more?

I begin with the explanation the Colorado Department of Education (CDE) gave for this change, when I wrote AV#168, “ELA scores hide the gap.” Then I continue my chronological review, from 2002 to 2018.

What I heard from the Assessment Office at CDE - 2017

In the summer of 2017 I asked CDE’s Assessment Office why the state no longer provided the same information on reading and writing we had seen for over 15 years.  Here is part of the first response:

“CSAP/TCAP had separate reading and writing tests with established performance levels, which is why the state posted proficient and advanced for them separately. CMAS has a single ELA test with established performance levels; so percent met and exceeded expectations is available for ELA only.… Subscore information is available through the school and district reports… These reports are not public, since they include student information.” (7/28/2017)

I then pestered Joyce Zurkowski, who continues to lead the Assessment Unit at CDE, with numerous questions. She was helpful and forthright in her answers. Keep in mind the following points she made as you read the rest of this newsletter. Do they leave you, too, with more questions? Here are her words:

1.      “Below is the timeline for implementation and elimination of the writing assessments.
AV#168, “ELA scores hide the gap: time for the truth on reading and writing scores,” Oct. 3, 2017, includes more of CDE’s response to my questions at that time. (At Another View's website)
1997:     4th grade                                                  
1999:     7th grade
2001:     10th grade
2002:     3rd, 5th, 6th, 8th and 9th grades
2014:     last administration of writing and reading assessments   (Bold mine)
2015:     first administration of English language arts assessments”  (email 9/5/17)

2.      “We have not posted subscores historically. Under CMAS, those subscores include reading and writing, along with physical science, life science, and earth systems for science and other subscores for math and social studies. We do not believe subscores should be used for high stakes policy decisions…  With that said, yes, we believe educators should have access to subscores, including the reading and writing subscores under CMAS, and they do.” (email 9/19/17)

3.      My email to her: “Reading and writing are two of the central standards, so this is really not like subscores for ‘physical science, life science, and earth systems for science.’” (email 9/20/17) 
Her response: “Actually, they are parallel. The Reading, Writing and Communicating standards have four standards: Oral Expression and Listening, Reading for All Purposes, Writing and Composition, and Research and Reasoning. There are three science standards as you identified above.”

4.      In that same email I sent her: “At present I think the state is not being as forthcoming as it should be in presenting vital information it has re student performance on these two separate and critical standards.  ELA results actually mask something you know, and we should all know.”
Her response: “The legislature made the decision to stop having separate reading and writing assessments and move to an ELA assessment.” (email 9/20/17)  (Bold mine)


2002 – 2006 – Teaching two of the three r’s (readin’, ‘ritin’, & ‘rithmetic) – Back in the classroom

CSAP - % Proficient or Advanced – 7th grade
Parker Core Knowledge Charter School
Grade 7
reading
writing
gap
1999
85
70
15 % pts
2000
94
72
22 % pts
2001
82
58
24 % pts

I taught English for 10 years in New England in the 1970’s and 80’s. I was glad to return to the classroom in the fall of 2001, teaching grades 7 & 8 at Parker Core Knowledge (PCK), a K-8 school. If I had been smarter, I would have taken note of the recent performance of students at my new school. (See box.)


Our task as Language Arts teachers is to address two standards. I depended a good deal, as a “new” teacher, on the state’s 17-page Colorado Model Content Standards for Reading and Writing. (Developed by the 24 members of the Colorado Model Reading and Writing Standards Task Force.) Page 4 lists the six specific standards: #1, #4, #5, and #6 dealt with reading; #2 and #3 addressed writing. As a Core Knowledge school, PCK had a detailed curriculum guide that clearly met those standards in reading. We worked hard to create a curriculum that would address the writing standards just as well.

Each March—CSAP. I handed out the CSAP test booklet that addressed both the reading and writing standards. Six testing sessions, an hour each, with almost equal time for the reading and the writing sections. There were also three sessions of an hour each for the math portion. A total of nine hours of testing, with essentially three hours set aside for each of the three key content standards.

CSAP - % Proficient or Advanced – 7th grade
PCK Charter School
Reading
Writing
2002
91
86
2003
95
91
2004
95
91
2005
98
93
2006
95
95
Good data matters. Especially if you use it! We received the CSAP results in August and examined the scores for each content area. Over five years we were able to close that gap in reading and writing. (See box.) As I was the only middle school English teacher at PCK, my 7th graders from the previous year were now my incoming 8th graders. We had a second chance to help those still partially proficient in writing make good progress.   



2008 - 2010

In 2008 the state began to revise its standards into ten “content areas”; one area was now called Reading, Writing, and Communicating.[iii] Colorado also reviewed the newly released Common Core State Standards (CCSS) in English language arts and mathematics. In August 2010 the state board adopted these as part of the  new Colorado Academic Standards. CCSS used the terms “English Language Arts Standards,” or “ELA/literacy” standards. Did a shift in nomenclature change our focus? Opponents of the Common Core accuse it of all kinds of shenanigans, but CCSS is not to blame here. Its ELA standards are all about reading and writing. Anyone who questions the “different skills” I speak of, please look at the CCSS paragraphs on reading and writing in the Endnotes[iv]. Or compare its five pages on Reading Standards for Literature and Informational Text 6-12 with its six pages on Writing Standards 6-12.[v] I rest my case.

Furthermore, if some of the terminology changed, CDE remained committed to those standards for another five years. By the end of 2010 it produced Grade level specific Reading, Writing, and Communicating standards for grades K-12.”[vi] One example: for grade 7, five pages on Reading for all Purposes, three pages on Writing and Composition. Good stuff—especially for new teachers.

Gov. Bill Ritter signed the Education Accountability Act of 2009, which reinforced the state’s commitment to report the reading, writing, and mathematics results each year. It added clarity and emphasis to the growth model, with a focus on “how much growth the students actually make in one year toward proficiency on state content standards” (Legislative declaration – 22-11-102).

The law also stated, in (22-11-204) Performance indicators - measures[vii]:

(I) For each student enrolled in a public school in the state, the department shall determine the student's achievement level in the subjects included in the statewide assessments, as demonstrated by the score achieved by the student on the statewide assessments. …
(II – IV)) For each public school… For each school district … For the state, the department shall calculate the percentage of students enrolled … who score at each of the achievement levels on the statewide assessments in each of the subjects included in the statewide assessments.

Back then we all agreed: “each of the subjects included” meant … reading and writing.

The Education Accountability Act of 2009 sought “information concerning performance …[at each level: state, school district, and school] … that is perceived by educators, parents, and students as fair, balanced, cumulative, credible, and useful.”[viii] That last word is vital for English teachers, as it is for schools eager to see how well their students performed on these two separate standards: reading and writing. Such distinctions are not only useful: if we are measuring separate skills, they are necessary.  

Colorado stayed the course.  Statewide assessments were given (CSAP through 2011; TCAP 2012-14) on each of the core content standards: reading, writing, and math. CDE presented the results to parents and the public. And those scores (see below) revealed the on-going gap in reading and writing skills.


Colorado – grades 3-5 - % proficient/advanced in reading versus writing (2007-2014)[ix]


Grade 3
Grade 4
Grade 5

reading
writing
gap
reading
writing
gap
reading
writing
gap
2007(CSAP)
71
54
17 % pts
64
49
15 % pts
69
57
12 % pts
2008(CSAP)
70
50
20 % pts
66
52
14 % pts
70
59
11 % pts
2009(CSAP)
73
54
19 % pts
65
51
14 % pts
69
58
11 % pts
2010(CSAP)
70
50
20 % pts
66
50
16 % pts
70
57
13 % pts
2011(CSAP)
73
51
22 % pts
65
56
9 % pts
69
60
9 % pts
2012(TCAP)
74
52
22 % pts
67
49
18 % pts
69
58
11 % pts
2013(TCAP
73
51
22 % pts
68
56
12 % pts
70
57
13 % pts
2014(TCAP)
72
51
21 % pts
67
52
15 % pts
71
55
16 % pts


2014 – CDE on Reading and Writing as part of the Colorado Model Content Standards

As late as September 2014, in its report on the Transitional Colorado Assessment Program[x], the Colorado Department of Education continued to list Reading and Writing as part of the Colorado Model Content Standards. When there was an explanation of subcontent areas, it listed “fiction, nonfiction, vocabulary, poetry,” etc. No one then was talking about Reading and Writing as subcontent areas, as subscores, or sub anything. Why would they? They were content standards. (CDE’s 2019 Fact Sheet on Literacy Standards[xi] indicates that the Office of Standards still sees both as standards. True?)


2015-2018 – PARCC/CMAS and ELA scores: Not Meeting Expectations

To explain when—if not why—a fundamental change took place after 2014, the best evidence I can find are the revisions in statutes made by the General Assembly in 2015. I present a comparison, 2010 versus 2015, in Addendum A, for those interested. A few tweaks in wording, with huge consequences.

2018 CMAS[xii]
English Language Arts/Literacy

% Met or Exceeded Expectations
Grade 3
40.4%
Grade 4
46.1%
Grade 5
47.4%
As a result, if I have this right, English Language Arts has supplanted reading and writing—not just in our terminology, but in what gets reported. CDE’s Assessment Office tells us reading and writing results are subscores for the ELA test, much like physics or life science are subscores for the science assessment. Who would have thought, in 1993, as we began the standards effort with the “first priority state model content standards as reading, writing …,”[xiii] that these two standards would disappear under an amorphous term like English Language Arts?

In AV# 168, when I called these developments “a terrible step backwards,” few agreed. And yet here in April 2019 we are revising the READ Act (SB19-199) without knowing what percentage of our 3rd and 4th graders are meeting expectations in reading.  Or in writing. (See Addendum B.)

And here in April 2019 we are working on changes in the accountability system (HB18-1355)—an important, complex, and painfully wonky effort—while ignoring such simple questions as the two with which I began this pair of newsletters: How well can they read? How well can they write?

English Language Arts results do not give us the answers we need.




Addendum A – from Standards (in Reading and Writing) to English Language Arts


2010
2015

Colorado Revised Statutes Title 22 Education § 22-7-1006 Preschool through elementary and secondary education--aligned assessments—adoption—revisions

(1)(a) On or before December 15, 2010, or as soon thereafter as fiscally practicable, the state board shall adopt a system of assessments that are aligned with the preschool through elementary and secondary education standards and are designed to measure students' levels of attainment of the standards and to longitudinally measure students' academic progress toward attaining the standards and toward attaining postsecondary and workforce readiness.                       (Bold mine.)

Colorado Revised Statutes Title 22 Education § 22-7-10063 State assessments—administration—rules

(1)(a) Beginning in the 2015-16 school year, the department of education, in collaboration with local education providers, shall administer the state assessments in the instructional areas of English language arts, mathematics, science, and social studies, as adopted by the state board pursuant to section 22-7-1006 , as follows:

(I) The department shall administer a state assessment in English language arts and a state assessment in mathematics to all students enrolled in grades three through eight in public schools throughout the state.
(II) The department shall administer a state assessment in science to students enrolled in public elementary, middle, and high schools throughout the state.
(III) The department shall administer a state assessment in social studies to students enrolled in public elementary, middle, and high schools throughout the state.   (Bold mine.)



Addendum B – Comparing READ ACT figures to ELA scores, grade 3. Leading me to suspect …

Here is one more reason we need a breakdown on reading versus writing scores. While the number and percentage of third grade students found to be Significantly Reading Deficient (SRD) on the READ Act assessments (not surprisingly) closely matches those scoring in the lowest category on the ELA test, that still leaves over 40% of the third graders who scored Partially or Approaching Meeting Expectations on the ELA assessment. In that latter group especially, CSAP/TCAP results suggest many of these students are meeting expectations as readers, but not as writers, and thus their overall score on ELA fell below the minimum needed to Meet Expectations. Would it not help to know if a good many of these third graders are, as I suspect, reading at grade level, but are not there yet with their writing skills?

3rd grade –% and # SRD - and % and # in each of the performance levels for CMAS/ELA – same year

SRD
CMAS/English Language Arts – Performance Levels

%/# of students identified with SRD
%/# Did Not Yet Meet Expectations
%/# Partially Met Expectations
%/# Approached Expectations
% Met/Exceeded Expectations
2017
18.2% - 12,251
18.6% - 11,831
17.5% - 11,131
23.8 – 15,139
40.1%
2016
17.2% - 11,560
19% - 12,043
19.4% - 12,297
24.3 – 15,403
37.4%
2015
16.1% - 10,639
19.6% - 12,284
19% - 11,908
23.2 – 14,540
38.2%



Endnotes



[iii]2009/2010 Colorado Academic Standards include 10 content areas for preschool through 12th grade (comprehensive health; dance; drama and theater arts; mathematics; music; physical education; reading, writing and communicating; science; social studies; visual arts; and world languages.  http://www.cde.state.co.us/standardsandinstruction/2009standards
[iv] Common Core State Standards – Key Features of the Standards
-          Reading: Text complexity and the growth of comprehension
The Reading standards place equal emphasis on the sophistication of what students read and the skill with which they read. Standard 10 defines a grade-by-grade “staircase” of increasing text complexity that rises from beginning reading to the college and career readiness level. Whatever they are reading, students must also show a steadily growing ability to discern more from and make fuller use of text, including making an increasing number of connections among ideas and between texts, considering a wider range of textual evidence, and becoming more sensitive to inconsistencies, ambiguities, and poor reasoning in texts.

-          Writing: Text types, responding to reading, and research
The Standards acknowledge the fact that whereas some writing skills, such as the ability to plan, revise, edit, and publish, are applicable to many types of writing, other skills are more properly defined in terms of specific writing types: arguments, informative/explanatory texts, and narratives. Standard 9 stresses the importance of the writing-reading connection by requiring students to draw upon and write about evidence from literary and informational texts. Because of the centrality of writing to most forms of inquiry, research standards are prominently included in this strand, though skills important to research are infused throughout the document.

[viii] Education Accountability Act of 2009 – Legislative declaration 22-11-102.
[ix] SCORES - 2007 from “CSAP Roundup – State officials hope flat results spark changes,” The Denver Post, Aug. 1, 2007; CSAP/TCAP SCORES - 2008-2014 from CDE web site, https://www.cde.state.co.us/assessment/coassess-dataandresults. GAP – my own math.
[xi] Colorado’s Literacy Standards: Key Points, http://www.cde.state.co.us/communications/ccssela
[xiii] State law (22-7-405) (1)(a). "Powers and duties of the state standards and assessments development and implementation council."