AV #257 - How well do our students write? Why don’t we know—as we did, 1999-2014?

 

ELA scores do not help the public see if students’ writing skills meet grade level expectations.

CDE reports “English Language Arts” results, which hides differences in reading and writing.

 

From 1999-2014 Colorado had useful data. Writing skills mattered. Who talks about writing now?

 

In April, over 300,000 Colorado students (grades 3-8) will take the CMAS. It is not—please note—all multiple choice. The Language Arts section includes three writing tasks. But what will we see next August? ELA scores.

Once upon a time (remember?) writing made the headlines. Reports spoke of writing. Not of an acronym.

Owens frowns at school scores

Results ‘would be shocking if they weren’t so sad,’ he says of latest reading, writing tests

(Rocky Mountain News, Sept. 30, 1999)

                                          Gender gap in writing tests favors girls                            1999

(The Denver Post, Sept. 30, 1999)      

    

    Stress on writing demands clarity

(from The CSAP Challenge – 24-page Special Report)

“It’s not enough to answer ‘64.’ You have to explain how you got there. After reading a passage, the main idea won’t rest next to A, B, C, or D. You have to come up with your own words…

“The Colorado Student Assessment Program [CSAP] requires writing, writing, and more writing, and not just on the writing section itself.  (The Denver Post, July 26, 1999) 

                                   Biggest metro-area gains                                                                                     2006

Fifth-Grade Writing                                      2005                 2006                                           

[School and District]

Pennington ES  (Jeffco)                                               25                         75                         

Teller ES (DPS)                                                             33                         69

Rocky Mountain ES (Jeffco)                                       59                         94                         

Academy of Charter Schools (Adams 12)                45                          79

                                                             (Rocky Mountain News, Aug. 3, 2006)

 

          DPS matches record gains in reading, sees strong gains in writing and math

… DPS outpeformed the state in growth in all 16 reading and writing tests…

                                                        (Rocky Mountain News, July 29, 2008)                                                            2008 

CSAP: Drop in writing scores spurs concern

                                             (The Denver Post, August 11, 2010)                                                                 2010

       **

Remember the 3 R’s?  Reading, and now math, are in the spotlight. Education Week (Jan. 26, 2023 issue) had some fun with its cover page of a special report Just these six “words.”

                   The Science of Reading

                                                                            ^

                                                                        and writ-

As if to say, are we forgetting something? 

A. Why say there are differences between how well students read and how well they write?

Evidence

1.  For 16 years (CSAP/TCAP years), we saw scores like these, below.

Colorado’s Student Assessment Program (CSAP) first tested grades 4 and 7 in reading and writing in the spring of 1999. In time it was every grade. By August, the results were public: a reading score, and a writing score. Often front page news. Here are the different results for five of those years, for two grades.

                          % Proficient or Advanced on CSAP – Grades 4 & 7 – 1999 - 2014[i]

COLORADO	Grade 4			Grade 7
	reading	writing	Gap % pts	reading	writing	Gap % pts
1999 (CSAP)	59	34	25	56	41	15
2001 (CSAP)	63	46	17	63	52	11
2006 (CSAP)	68	50	18	64	56	8
2010 (CSAP)	66	50	16	68	58	10
2014 (TCAP)	67	52	15	69	61	8

 

What can we make of the striking improvement in writing scores, even better than in reading?    

My takeaway: Provide the data, and we pay attention. It can impact what schools and teachers prioritize. My fear: No public data on writing since 2014 can impact … what we do not prioritize.

 

(From 2002-2006 I taught English to grades 7 and 8. CSAP’s reports on the writing skills (vs. reading skills) of our students demanded that we address the gap. We had good results. See Addendum A.)

2. Reading/Writing gaps in two other states in 2022.

We changed our state assessment in 2015. The Colorado Measures of Academic Success produces results that obscure any differences in students’ reading and writing skills. The Colorado Department of Education releases CMAS results in August.

What does the general public see? A Math score and an ELA/literacy score.

Reading skills, nothing. Writing skills, nada.

(CDE would rightly say districts, schools, and parents do get a breakdown. More on that later.)  Nothing like what CDE reported from 1999 to 2014

Several states, though, still assess and report on students’ writing skills. This is another reason I suspect the gaps in how well Colorado’s students read and write have continued. We see this pattern elsewhere. Why would we be different? 

KENTUCKY – Performance Levels - % Proficient / Distinguished - 2022[ii]

	Reading	Editing & Mechanics	Writing On-Demand	Combined Writing	GAP – Reading vs.  Combined Writing
Elementry Schools	45	47	32	37	8 % pts
Middle Schools	43	45	38	39	4 % pts
High Schools	44	46	36	38	6 % pts
Average – EMH	44		35
Note, too, the gap between Reading vs. Writing On-demand

VIRGINIA – Pass rates – 2021 and 2022[iii]

Not long ago Virginia gave a separate writing test for grades 5, 8, and 11. As part of an effort to “reduce the testing burden,”[iv] the 5^th grade test was dropped. It is still given to grades 8 and 11.

 

	2021			2022
PASS RATES	Reading	Writing	Gap - % pts.	Reading	Writing	Gap - % pts.
Grade 8	69	54	-15	72	57	-15
Grade 11	81	76	-5	85	74	-11
Economically Disadvantaged				59	49	-10
Black				60	47	-13
English Learners				32	18	-14

Kentucky and Virginia know something about their students that Colorado does not. Wouldn’t we like to have this information?

3. Teaching English for 18 years, I saw a countless number of students who did not write well.

Even those who were good readers. OK, hardly “evidence.” Why not ask English teachers you know? Do they see a difference? (And if you know them well enough, you might ask how many students they teach, and if they feel the workload affords them the time to address their students’ writing skills as much as is needed. I am sure English teachers in Denver and Aurora who tell me of the 170 students enrolled in their classes would be quick to say, absolutely not!)

SAT: What the Writing and Language Test Questions Are Like “Each passage has 11 multiple-choice questions. The questions fall into two main types: those where you improve the expression of ideas, and those where you have to recognize and correct errors in sentence structure, grammar, usage, and punctuation.”[v]

High school writing  

For the past decade I have worked with many high school juniors and seniors (at College Track), often helping students craft their college essays. It has shed light on their writing skills. As individuals, yes, they seem “college-ready.” But as writers? I worry.

 But we have no reason to ask the state to tell us more about the writing skills of our high school students. Why? This may surprise you. Because the College Board tests, the PSAT 9, PSAT 10, and SAT (grade 11) that we use to measure academic achievement in our high schools, do not ask students to write one sentence. It is all multiple choice. In the end, students receive two scores, one for Math, and one for “Evidence-based Reading and Writing.” But who thinks this is truly a “writing score”?

 

What TEMPLE GRANDIN, Colorado State University professor - and prolific author, sees from her graduate students. A complaint, and (implied) a suggestion.     The other thing that we’re not teaching students today is writing skills. Graduate students have terrible writing skills now … and I’m not the only professor right now complaining about writing skills. The way I learned to write was my work was marked up and I had to correct it. And I’m finding some of my students, smart students, never hardly had to write a term paper, never had anybody correct the grammar in their papers.[vi]

(We may not even want to know their writing skills. The final TCAP in 2014 included writing tests for grades 9 & 10. See Addendum B.) 

Ironic, isn’t it, that College Board tests do not ask for any writing from high school students, whereas CMAS expects even third graders to write.

 So with little there, I have focused on the CMAS and students in grades 3-8. At least this test gives us a chance to learn if most of these students can write well.

 

B.  Where do we see the terms we use, and perhaps the focus, change?

Colorado adopted the Common Core State Standards (CCSS) in 2010. That was a turning point.

BEFORE: Throughout most of 1999-2014, Colorado schools looked to the Colorado Department of Education’s standards for guidance. This old English teacher still has his Standards For Reading and Writing (2000). Our assessments (CSAP/TCAP), focused on the three primary standards: reading, writing, and math. 

AFTER:  CCSS brought us new terminology: English language arts (or ELA).  

“After Colorado adopted the Colorado Academic Standards, students began taking the Colorado Measures of Academic Success (CMAS) assessments in 2014 for science and social studies and 2015 for mathematics and English language arts/literacy.”  (CDE – CSAP/TCAP[vii])

Ever since, CDE uses this term for state scores: “CMAS Math and ELA State Summary Results.”

W-r-i-t-i-n-g has disappeared. ELA rules.

C. “Conversations” with the Colorado Department of Education – what can we know?

Given: a) how important I believe it is that we know how well our students write,

             b) how much writing (that over 300,000) students actually do on the CMAS, and

  c) the enormous time and expense of this assessment,

I have asked CDE if it could not produce more useful information about our students’ performance than an ELA score. In AV #168 and AV #193 I shared our 2017 exchange where CDE had to correct what it first told me. In the end it made 2017 subscores for both reading, and for writing, available. CDE was always thoughtful and courteous. But I was not satisfied. 

This winter, I asked CDE’s Joyce Zurkowski, Chief Assessment Officer, for an update. Some of our exchange follows.

Reader, free to abandon this exchange as soon as you feel it goes too far into the weeds. Although that is partly my point. When parsing “subclaims” and “subscores,” I have lost my way. I want to find out how well students write. Isn’t that what the public wants to know?

PH       Back in 2017 Stephanie Boyd provided … data - subscores for reading and for writing - from the ELA assessment. [I included that 2017 information in this email.]

I assume such data is still publicly available. Could you please provide the subscores for each grade in 2019 and 2022?

JZ - The last time we provided both reading and writing subscores was in 2017. Starting in 2018, the test was shortened. One of the test characteristics that we had to give up was having both of those subscores.             

Historically, we have not provided subscores publicly because subscoresby design have lower reliability and we do not want folks making high stakes comparisons and decisions based on them. Given all that, [here - see table] are the reading scale scores for 2019 and 2022.

Grade	CMAS Reading Scale Score  - 110-190
	2019	2022
3	146.2	145.0
4	148.3	146.2
5	148.9	148.3
6	147.5	146.8
7	148.4	146.4
8	148.3	147.1

            PH  - Thanks very much Joyce. I appreciate this. Just one more question for now. Should I assume that the one-pager (front and back) that parents used to get – [the reading data and the writing data] – is different too? Or is there something on writing results from the ELA test that parents see?    

            JZ  - Peter, parents receive their student’s percent of points earned information for writing overall, language and conventions, and written expression. Mean percent of points earned is also provided for the district and state, as well as how students who are at the lowest end of “met expectations” scored. Because this information is not scaled, it cannot be compared across grades, years or other subclaims.

She also shared an example of what a parent receives. (Available online at CDE’s “Interpretive Guide to Assessment Reports – A Guide for Parents and Educators.”[viii])

 

	Possible points		0%	25%	50%	75%	100%
Writing			This is my summary of what a parent sees here. This section presents graphs to match the student’s scores. It includes a bar showing where the student’s scores put him or her on the continuum 0-100%, and beside it, a bar showing the state average for that grade.  See N. The graph also shows a line marking what level/score is needed to be “on track for the next grade level.”
Overall Writing Overall is calculated by multiplying Written Expression by three and adding Language and Convention points.	27	100%
Written Expression Student compose well-developed writing details from what they have read.	7	100%
Language and Conventions Students demonstrate knowledge of conventions and other important elements of language.	6	100%

 I studied that. The writing subscores were right there.   

I asked a principal to show me an example from 2022. There we see the state average, as clear as day. For each Writing category: Overall, Written Expression, and Language and Conventions. 

So I wrote Zurkowski again, pointing out the contradiction. I received an apology. “I should have made a distinction,” she wrote, “between subscale information versus subscore information.”

She went on to give a detailed explanation. Her concluding note was encouraging. She wrote that as CDE continues “to encourage the evolution of school, district, and state assessment systems,” it will evaluate what “level of information” can be made available.

Lost in the weeds?

I am sorry for belaboring the point. Besides, I have zero expertise in assessment; some of what Zurkowski has kindly provided me is confusing. I trust those of you more knowledgeable than I am will be able to pursue the question: what can the state tell us? What I am told feels more convoluted than it needs to be, but this may be my fault.

My wish is that the Colorado Department of Education decides to find out how much it can share with the public, rather than what it can't share.

The three R’s. With math—results are clear. We need that clarity with reading, and writing.

Once again, ELA is not enough.

 

Addendum A

 2001 – 2006 – 7^th grade writing, one school

No Child Left Behind became law in 2001, requiring all states to measure reading and math. By then, however, Colorado had already decided to include a writing assessment. Grades 4 and 7 at first, eventually grades 3-10.

 CSAP - Writing scores in our school, 2001-2006 – Good data helps you focus 

Our school was still beginning to establish its middle school program in 2001. I became the English teacher for grades 7 and 8 that fall. I looked at the scores for 7^th graders from the previous spring, 2001.

 

2001 - % Proficient/Advanced:  Reading-82%  /  Writing-54%.

 I found the scores useful. They gave us an objective report on our students’ writing skills.

CSAP - % Proficient or Advanced  7th grade
Parker Core Knowledge Charter School
Grade 7	Reading	Writing	Gap
2001	82	58	24 % pts
2002	91	86	5 % pts
2004	95	91	4 % pts
2006	95	95	-

Given my background in teaching writing in two high schools and the bright, motivated students we had, we saw improvement right away. The school—I was hardly alone in this—made writing a priority. Over my five years, 7^th grade scores kept climbing.  

 

An artifact from 2004 – when we had the data

I am fascinated to hold in my hands the two-pager giving the Writing scores for grade 7—my class-for 2002, 2003, and 2004 – and for 36 other Core Knowledge schools. A report like this went out to all of us. Two pages of CSAP Writing scores, for grades 4-8, at each school.

In 2002 seventh graders in my two classes at Parker Core Knowledge Charter were tied for third on the list, 86% proficient or advanced in Writing. Two years later our seventh grade was fourth, but the percent meeting expectations was even better: 91% proficient or advanced in Writing.

It wasn’t competitive. At least I don’t think so. Although – mmm - I notice I highlighted every school with better scores than we had!  Anyway, we had good data, and we paid attention.

Colorado Core Knowledge Schools - 7th Grade CSAP WRITING Scores - 2004

1.     Cheyenne Mountain Charter Academy – 95%

2.     Littleton Academy – 94%

3.     Liberty Common School – 92%

4.      Parker Core Knowledge School- 91%

Addendum B

High School – Writing

From 2014, the last time Colorado tested students in writing grades 3-10.

TCAP Writing scores for secondary students. % Proficient/ Advanced

 

Grade 7	Grade 8	Grade 9	Grade 10
61%	56%	54%	49%

Was the TCAP well-suited for our lowest-performing high schools? I doubt it. But when a state assessment tells us that less than one-quarter of 10^th graders in certain schools (see below) are proficient in writing , it serves, at least, as a wake-up call. When the achievement gaps are so clear, district and school leaders will feel compelled to address the issue. Or so we hope.

As principal told me recently, “what we publicize, we emphasize.”

2014 – TCAP - % Proficient/Advanced in Writing

STATE of Colorado –  Average – 49%

Gateway (APS)                                 23%

Westminster High                           21%

Manual (DPS)                                   17%      

Aurora Central (APS)                    17%

Abraham Lincoln (DPS)               11%

One more question: Have we given up on finding some way to measure how well our high school students write?

 

Endnotes

[i] Colorado Department of Education, CSAP/TCAP scores - http://www.cde.state.co.us/cde-search-results?search_api_views_fulltext=CSAP+scores&search_api_views_fulltext_op=AND.

[ii] Kentucky Teacher, Kentucky Department of Education –

https://www.kentuckyteacher.org/news/2022/10/kde-releases-2021-2022-school-report-card-data/.

[iii] Virginia Department of Education, “SOL Test Pass Rates & Other Results” - https://www.doe.virginia.gov/data-policy-funding/data-reports/statistics-reports/sol-test-pass-rates-other-results.

[iv] Phone call with Jason Ellis, Virginia Department of Education, March 8, 2023.

[v] College Board - https://satsuite.collegeboard.org/sat/whats-on-the-test/writing-language

[vi] Interview with Dr. Temple Grandin, Professor of Animal Science at Colorado State University and prolific author - including “Visual Learning” (2022), which led to this conversation on Book TV, C-SPAN 2 (Nov. 14, 2022). Hosted by Smithsonian Associates. https://www.c-span.org/video/?523916-1/visual-thinking

[vii] CDE - CSAP/TCAP - https://www.cde.state.co.us/assessment/coassess

[viii] CDE - “Interpretive Guide to Assessment Reports – A Guide for Parents and Educators” - See Page 11: English Language Arts Proficiency, “Sample Individual Student Performance Report, CMAS ELA and CSLA.” https://www.cde.state.co.us/assessment/cmas_coalt_interpretiveguide_2022.