The Myths of Standardized Testing

“You’ll never have the kind of schools you would like to have, nor the test scores you want, unless you do something about ______.”

 By David Berliner, Professor Emeritus, Arizona State University

Dr. Berliner’s notes from a presentation at the Annual Meeting of the Horace Mann League, on Friday, February 15, 2019.

Thank you all for coming today. I was asked to provide about a presentations on things I think about, that might also be of interest,…. maybe even useful to you all! Feel free to ask questions any time.

I am well aware that you folks do all the hard work, while I have the luxury of being at a university, away from the actual hard work of educating our youth. But before her retirement my wife was a public-school teacher and principal,… my sister-in-law was also a school teacher and principal,…. both my son and his wife are in higher education, … and my daughter is an educational researcher. All these close family relations insure that I do not become another pointy headed academic!

The title of what I have put together for your consideration is “You’ll never have the kind of schools you would like to have nor the test scores you want unless you do something about ___X___.” I’ll relate this to testing issues, as advertised, but as I prepared I strayed a bit from the advertised topic because test scores are related to so many factors other than the effects of teachers and administrators.

First, I want to argue that the state and district environment—its care and nurturance of its citizens and educators—matters a lot. These factors dramatically affect standardized test scores and many of the achievement outcomes that we value. Let’s run a little thought experiment to illustrate this. I am going to ask you which of two states is likely to do better on the reading and mathematics NAEP tests—our nations report card. I will present data to you about these two states, state A and state B. 

OK, any predictions about the states’ score on NAEP? Anyone picking state B?

         I assume that no one here is surprised.  

Anyone here know which states these actually are?

So, my thinking is this:

You’ll never have the kind of schools and test scores you would like to have unless you do something about making your state a better place to live in,… to work in… and in which to raise children. If you just hunker down to address school issues you may be failing many of the children you care about and for whom you are responsible. In today’s America you need to fight as hard for taxes to support healthy communities, families, and schools, as you do for the paper needed by the copy machine, and for professional development days. The non-political, or a-political school administrator, must become a remnant of the past.

Really–I am not a crazy liberal activist! What I am simply stating should be obvious: A lot more time must be spent in political discourse—above and beyond your school board. This seems to me to be a necessary job requirement in contemporary times. The recent trend in which a lot of teachers ran for office, and were elected, makes it clear that I am not alone to suggest what needs to be done.

I also have data to back up what I suggest. I know, and you should too, that the richness of the environment in which we raise our children improves not just their achievement, but their IQ! Let me share one such set of data

Why did IQ—which is usually pretty stable, and rarely changes so much in a population over a mere decade—actually rise 10 points—a point a year!?

Solicit Answers. Discuss

ANS: Electricity/newspapers/strangers who were engineers/ state capitol and federal linkages with local communities/ different and more cognitively demanding kinds of jobs for adults/ etc.

My Point is this: wealthier,.. healthier,…and less parochial environments promote cognitive growth.

You need to think about all the ways your community could provide a richer environment for its kids so that their IQs and their achievement test scores can be whatever their genetic makeup promotes them to be, …You don’t want your students’ test scores to be restricted by family and neighborhood circumstances!

Does that happen? You bet it does. A number of convincing studies suggests that genes do not express nearly as much in “poor” childhood environments– environments with food insecurity, inadequate parenting, evenings spent in TV watching or computer gaming, unsuitable neighborhoods, etc. On the other hand, “rich” environments allow the full expression of one’s genetics… whether it be corn yields or the talents of our youth. It’s not that poor and rich kids necessarily have different IQs, it’s that among rich kids their genetic IQ is more likely to be expressed, whatever it is, while for poor kids, in less than ideal environments, genetic IQ is less likely to be made manifest. Your district’s standardized test scores will reflect that restriction in the expression of intelligence in many of your students’.

This suggests to me that every school administrator needs to understand that housing in their community is an educational issue not just a social issue. Housing patterns—the social environments in which our youth are raised –strongly influence the test scores your schools will display. So,…. I think that in many districts “You’ll never have the kind of schools and the kinds of test scores you might want unless you do something about your community’s housing patterns.”

It’s no secret: our children are tracked into different neighborhoods on the basis of their family’s income, ethnicity, and race. This is where many of our school problems begin. We seem deliberately blind to the fact that housing policies that promote that kind of segregation are educational policies, as well.

When we allow overwhelmingly wealthy, middle-class, and poor neighborhoods to develop, we destroy the chance for the local neighborhood school to help better all our children by bringing diverse income, racial, and ethnic groups together. If they can be brought under one roof the ordinarily beneficial middle-class educational norms are likely to dominate school culture. The cohort you go to school with influences your scores on standardized tests. The famous Coleman report—now 50 years old– showed us that schools were not as powerful as we had hoped they would be: families and neighborhoods had a powerful influence on the achievements of the kids we teach. But recent reanalyses of the Coleman report revealed that those researchers underestimated the power of the cohort with whom kids go to school. Who is in your school matters a lot, and local housing patterns have a big influence on that.

Neighborhood schools, affectionately supported in American folk beliefs as a great equalizer in the melting pot we think of as America,… now perform on school assessments almost exactly as that neighborhoods’ income predicts it will! The neighborhood school in a society with an apartheid-lite housing policy, like ours, is killing us!

In New York and Illinois, over 60 percent of black kids go to schools where 90-100 percent of the kids are nonwhite and mostly poor. In California, Texas and Rhode Island, 50 percent or more of Latino kids go to schools where 90-100 percent of the kids are also not white, and often poor. Similar statistics hold for American Indian kids. And throughout rural America there is almost always a “wrong-side-of-the-tracks” neighborhood, or a trailer park area, in which poorer people are expected to live. The kids in those neighborhoods generally go to schools with the other kids from those neighborhoods. It is properly thought of as an apartheid-lite system of housing. And the test scores that we see in those schools almost always reflect the housing patterns that exist, not the skills of teachers or the competency of the schools’ administrators.

So,… school administrators who are not heavily involved in their community’s housing policies are likely to promote, through neglect, an America most of us do not want. You all know that the percent of poverty in a school almost always informs us of that schools’ test scores. And if the test scores are used to assign letter grades for schools, as is done in some really stupid states, like my own state of Arizona, those letter grades will almost certainly be correlated with poverty. Regardless of how good the administration and teaching in a school actually is, it can be labeled a D school without anyone observing the quality of education provided in that school.

Let me share one of many examples of this high correlation of school district poverty rate, and test score data. This is Nebraska data.

High school districts in Nebraska, School poverty rate, Grade 11 reading and mathematics scores.

Poverty rate and scores on the Nebraska state tests suggest the tests need not be given since the scores are easily and accurately predicted from the poverty rates.   

Clearly school poverty rates tell us a lot about what test scores to expect. And then, in the states that are the dumbest and or the meanest, the test scores are used to determine the letter grades, as if poverty was not an issue. I can illustrate this issue using North Carolina data. Here we have the letter grades associated with percent poverty.

As I said, poverty determines your test scores and your letter grades. And it is foolish to take the bad rap for what is really a society that won’t do more to lower the rate of families in poverty.  

Data such as these makes me say again that housing is a political issue with which you need to be concerned if you care to promote democratic values. The evidence is overwhelming that the wealthier and school-smart kids loose little or nothing in tested ability when placed with poor kids who achieve less. At the same time, the poorer kids frequently do better on those tests than if they were in environments with other poor kids. Parents hate it when I present data like these, and they argue with me. But what I say is frequently true: Mixing social classes and doing away with tracking doesn’t really hurt advantaged kids, while it does advantage the kids we ordinarily think of as disadvantaged! 

Here is data that supports this claim:

Among fourth grade students:

  • For every 1 percent increase in middle-class classmates, low income students improved 0.64 percentage point in reading and 0.72 percentage point in math.
  • Any given low income student attending an 85% middle class school rather than a 45% middle class school saw “a 20 to 32 percentage point
  •  
  • David Rusk Study of Madison-Dane County, WI.int improvement in that low-income pupil’s test scores.”

Ok–even in the apartheid-lite system we have, is there some way to provide the necessary rich environments that helps kids to flourish? There sure is. It’s the promotion of high-quality early childhood programs. Thus…I say: You may never have the kind of schools and the test scores you would like to have unless you insure that low income children have access to high quality early childhood education.

What does the US look like compared to many other countries that recognize this fact?

And the richest, though not necessarily the wisest families know this:

These data made me think of Dewey, writing one hundred and fifteen years ago, in School and Society. He wrote,

 “What the best and wisest parent wants for his own child, that must the community want for all of its children. Anything less is unlovely, and left unchecked, destroys our democracy.”

So…. if parents who are the best educated and wealthiest want high quality preschool for their own kids, we should make such educational opportunities available for all our kids. Not to do so means that too many children of poor families will not profit as well as they might from enrollment in our public schools. Not providing high quality early education is also quite likely to hurt our always fragile democracy!

But what about the costs? What if we spent the huge amount of money necessary to have students of the poor receive high quality preschool experiences? If we did that here is what we can expect in return for that investment:

1, substantially reduced identification of these children as needing special education (that reduces school costs);

2. A much reduced achievement gap between kids in the lowest and highest social classes (that is good for democracy).

3. Reduced health problems throughout that individuals’ life (that is both humane and reduces society’s costs for health care)

4. Reduced dropout rates in high school (that has future tax savings for a community)

5. Increased high-school graduations rates (this also has future tax savings for a community)  

6. higher college attendance rates after high school (this has benefits for the local industries).

 7. Higher employment rates after high school (this increases tax revenues).

8.  Lower incarceration rates as adults (this lowers the costs to the community and state, as well as avoiding the personal tragedies for families with incarcerated relatives). And these 8 factors lead to point 9. Over 30 years high quality early education  provides a return on investment of around 10%.

So…If you live in a community with many poor kids, and you expect that community to still be around 30 years later, it is foolish, perhaps even mean spirited, not to invest in high quality early childhood education.

There is a related issue to address. It’s about what happens to low income kids, compared to high income kids, over summer.

So…. Id add this, You’ll never have the kind of schools you would like, or the test scores you desire, unless you do something about children’s summer school experiences. These should be less about the study of school subjects and more about enrichment, as often happens for the students of wealthier parents during the summer months.

Why do I add this?

Summer Learning & the Achievement Gap

K
Summer
1st
2nd
3rd
4th
Summer Reading Achievement
Trajectories
Low-Income Students
Middle-Income Students
Summer
Summer
Summer
Summer

Middle- and upper-class kids have a plethora of opportunities for leaning things in the summer that are school, as well as life related, and that also influences test scores. This wider range of experience gives them a better chance to read with greater comprehension. We know that reading comprehension, and a great deal of our understanding of social studies and science, is based on our experiences in the world in which we grow up. These experiences —on top of the formal school curriculum—make school subjects a lot easier to understand.  Here is what we know about life in contemporary America.

Clearly over the years the gap between wealthier and poorer students, in terms of their enrichment experiences while growing up— things like trips to museums, music lessons, trips to foreign countries, books purchased for them, tutors, and so forth–has grown greater and greater. This cultural and academic knowledge gap between richer and poorer students should be better attended to. It will never go away—but recognizing and addressing this issue is important.

I would add this to my message of how to improve test scores and our nation: You’ll never have the kind of schools or the test scores you would like unless you do something about absenteeism in your district. It’s really a no-brainer! If you don’t attend school you are likely not to learn what school offers. And what schools’ offer is linked (however loosely) with what is on the standardized tests, which are then used inappropriately to judge the quality of a districts’ schools and teachers. Is absence a problem? It sure is.

The first school listed here, the Morrisania school is located in the neighborhood I grew up in. Recent data showes that in this K-5 school, 85% of the kids are poor, and 42% of them have missed a month or more of schooling! If many of your kids miss a month or more of schooling, as is common all across the country, and their test scores are included in your schools’ data, you are being judged for instructional competency by means of a metric that cannot possibly be fair to you under conditions of high absenteeism. You are being judged with tests that assume the content of the tests was taught. The tests assume that students were exposed to the content. And if that is not true,…. how can you allow that to happen to yourselves? You either need to fix the absenteeism rates by devoting a lot of money to children and their families, probably by hiring many more social workers, or demand that those scores be removed from the data base that is used to judge a schools performance. Look at these data.

These data suggest that High Asian test scores and low American Indian and African American test scores may have a lot more to do with who actually shows up to school, rather than any alleged differences in ability!

Here is more on why social workers might be needed.

Finally, let me say what I expect all of you know too well: You’ll never have the kind of schools or the test scores you would like unless you do something about pay for qualified educational staff—teachers, bus drivers, counselors, librarians, nurses, social workers, and so forth.  Instead of administrators telling teachers and other staff not to strike, as was the case in Arizona and Los Angeles, they should be telling legislators and school board members that they cannot guarantee a high-quality educational experience for the children of their state with unqualified teachers and staff, and the resulting high levels of churn associated with that kind of staffing. Besides effects on achievement, teacher churn raises educational costs dramatically. The hiring of a new teacher can easily cost $15,000 per replaced teacher if replacements can be found. And in contemporary America that is harder and harder to accomplish.

Teachers leave the field for at least three reasons. Because they are not respected by politicians and newspapers, which demoralizes them; Because they burn our emotionally and physically since teaching is a lot harder than the public thinks it is; and because they do not make as much as other college grads in their states. By the way—NPR reported on Monday that Walmart truck drivers are now averaging $87,500 a year. At the same time average elementary teacher pay in the highest paying state in the union, New York, is about $80,000, and the average in my state of AZ is $44,000, half of what is earned by Arizona’s Walmart drivers.

\       

Here is a list of states that do and do not pay teachers a living wage. Remember, this is not a good wage. It is a living wage. It is what you need to survive, not thrive as a family. It’s the minimum income necessary for a worker to meet their basic needs such as food, housing, and clothing. The goal of a living wage is to allow a worker to afford a basic standard of living—and 30 states will not provide their teachers that minimum level of support. Here is more on teacher pay.

That’s a gap in 2017 of $339 a week, $1,356 per month, and $16,272 a year between teachers and other college grads. And the gains in salary made by teachers in inflation adjusted dollars, from 2000 to 2017, was negative in most states. In Indiana and Colorado teachers lost about 15% of their purchasing power. In North Carolina and Michigan they lost about 12% of their purchasing power. In my state of Arizona teachers lost about 10% of their purchasing power from 2000-2017. Our governor offered a 1% pay raise! With public support teachers walked out and won a 20% pay raise over the next 3 years. The result will give them the same purchasing power they had in 2000, and will not reduce the gap in pay vis-a-vis  other college graduates. 

 And you need to know that Teach for America teachers and alternatively certified teachers, both of whom are often paid less, and employed almost exclusively in districts that are underfunded, leave teaching at the highest rates. Thus, the poorest districts incur the highest costs for recruiting teachers for their classrooms. Recent data on turnover rates inform us that,… compared with regularly certified teachers,…. alternatively certified teachers are at 44% greater risk of abandoning their classrooms during the school year. And they are 152% more likely to leave the school at which they work at the end of the school year.  Teach for America teachers also abandon their classes (and their contracts) at very high rates both within the school year, and also at the end of their first school year. Indeed, at the end of their 5th year, only about 15% of TFA recruits continue to teach in the same low-income schools to which they were originally assigned.

Further, despite their claims, Teach for America teachers are not “better” teachers for low income kids. Though they may not be worse than other new teachers, they are not as good as experienced teachers who have been through a traditional teacher education program. Districts need to pay enough to keep experienced teachers for the simplest of reasons– they are a lot better at their job!

Experience matters and churn hurts schools. If America wants better schools as well as higher tests scores, America needs to pay higher wages and run schools that are fully staffed.

So now, as an old researcher, here is what I think: 1. Using state monies to subsidize charter and private schools is really a problem for me. They not only take away monies from the public schools—about 1 billion over the last decade in Arizona–but the vast majority of them won’t admit to their schools many of the kids that need special attention. Poor kids, special ed kids, and English language learners both reduce profits and lower test scores. So these schools take public money but will not serve the public, and therefor they are a malevolent force in our democracy. 2. I am deeply unhappy about our nations’ mindless commitment to high stakes testing, when everyone in research knows that outside of school factors play 6 times more of a role in determining classroom and school test scores than do the personnel at those schools. Nevertheless, if we want our public schools to be the best they can be, and their test scores to be higher than they are, than this is what I say:

You’ll never have the kind of schools and test scores you would like to have:

  1. Unless you do something about making your state a better place to live in, to work in and in which to raise children.
  • Unless you do something about your community’s housing patterns.
  • Unless you insure that low income children have access to high quality early childhood education.
  • Unless you do something about children’s summer school experiences. These should be less about the study of school subjects and more about enrichment, as often happens for wealthier students during the summer months.
  • Unless you do something about absenteeism in your district.
  • Unless you do something about pay for qualified educational staff—teachers, bus drivers, counselors, librarians, nurses, social workers, and so forth.