Saturday, February 22, 2020

How Much Academic Growth?

Recently, a colleague in England very politely called me on the carpet for something I wrote in 1981 that did not mercifully die in the interim. Here is his question.

Dear Dr. Glass

As you may know, the Education Endowment Foundation in England, in order to help with interpreting effect sizes in education, has suggested to teachers that an effect size of one standard deviation is approximately equal to one year’s progress, and they cite the following from page 103 of your 1981 book ‘Meta-analysis in education’ in support:

It is also known, as an empirical—not definitional—fact that the standard deviation of most achievement tests in elementary school is 1.0 grade-equivalent units; hence the effect size of one year’s instruction at the elementary school level is about +1, for example,

∆ = (4.0 - 3.0) / 1.0 = +1.
I asked one of your co-authors, Barry McGaw (whom I know well) if he could recall where this figure came from and he could not, hence this email to you.

The reason I ask is that for many of the tests currently in use in elementary schools in the US, one year’s progress is considerably less than one standard deviation, and for students in the upper elementary school (say grades 4 and 5) the available evidence, for example from Howard Bloom and his colleagues, suggest that one year’s typical progress is around 0.4 standard deviations.

Any light you can shed on this would be appreciated.

Sincerely, Dylan Wiliam
Dylan Wiliam BSc, BA, MSc, PhD, FRSA, AcSS, MBPsS
Emeritus Professor of Educational Assessment
University College London
20 Bedford Way
London WC1H 0AL

It took me a day or two to dig up some data pertinent to the question that Dylan asked. Thanks to the power of the world wide web, What was needed was near at hand.
Hi, Dylan
Sorry to be slow.

Yes, I was flying a bit blind in 1981 when I put that stuff in the book. I hope it was not ever used to anyone's disadvantage.

When I came up with "one year's growth is about 1.0 effect size," I was looking at a bunch of achievement test data in the early elementary grades that were easily available. What is pretty clear now is that "effect-size growth" for one year of schooling is about 1.0 in early grades (1-3) and steadily declines as you go up the grades. Because the st-dev of Reading is usually a bit greater than that for Math (for several reasons, some obvious), the effect-size growth is a little greater in Math and in Reading for one year's instruction for most grades.

Here are a couple of excerpts from available reports that report sufficient information to calculate effect-size growth for one year for several grades.
The first table is from a report by Michael Russell at Boston College. Here's the URL:

So, it looks like effect-size growth (ESG) is in the vicinity of 1 in the early grades and declines as the grades increase. And, Math ESG is a little bigger than Reading.

Here are more data showing the same trends.These data are from the norming of the Iowa Test of Basic Skills:

Let's look at one example. Math growth from Grade 6 to 7.

∆ = (250 - 232)/28.3 = .63

So, the average gain in Math achievement from the 6th to the 7th grade is about two-thirds of a standard deviation. In other words, the average 6th grade student moves from the 50th percentile to the 75th percentile of the 6th grade norms in one year. Also, the table shows the same trends that ones sees in other data: Smaller than 1.0 ESG in the intermediate grades; ESGs a little bigger in Math than in Reading.

I should have been more circumspect in 1980 when I was tossing off opinions without sufficient serious study. I was guilty of the same loose talk that I have so abhorred in the writings of people like Cohen who wrote that ES = .25 is "Small" and ES = .5 is "Medium," etc. It all depends!

Hope this helps.


I'm more sympathetic to those writers who leave their descendants instructions to burn everything they have written.

Gene V Glass San José State University

Arizona State University

University of Colorado Boulder, National Education Policy Center

The opinions expressed here are those of the author and do not represent the official position of San José State University, the National Education Policy Center, Arizona State University, or the University of Colorado Boulder.

Friday, July 20, 2018

"Fahrenheit 451" Lives!

Fahrenheit 451, published in 1953 in times with eerie similarities to today, is a dystopian novel by Ray Bradbury. The novel presents a future America in which books are outlawed and firemen burn any they can find. Somehow I am not all that surprised then when a proverbial "book burning" takes place right in on of my own home states of Arizona.

In 2014, David Berliner and I published a book under the title 50 Myths and Lies that Threaten America's Public Schools. We called out some very troubling trends that threaten to destroy the nation's last and greatest public institution, our public elementary and secondary schools. Trends like: privatization, vouchers, charter schools, tuition tax credits, wide-scale high stakes testing and test-score evaluation of teachers being pushed by a massive U.K.-based corporation. The book has enjoyed some visibility.

Apparently, some parents in an Arizona school district were less than impressed. It's no surprise. Arizona with its large Hispanic population and larger still aging white middle class and entrenched conservative establishment is a virtual bell-wether for the rest of the 49 states that will soon match it demographically.

The school district in question -- which shall remain nameless -- administers a summer reading program in connection with a high school AP English class. There is a reading list. The reading list became a problem.

Half-way through the summer, an administrator emailed the teachers in charge of the AP class with this message: " I wanted to take a minute to update you regarding three books on the English summer reading list: Columbine by Dave Cullen, The Smartest Kids in the World: And How They Got That Way by Amanda Ripley, and 50 Myths and Lies that Threaten America’s Public Schools by David Berliner and Gene Glass. There has been concern raised by parents at the district, that those three books may have a possible connection to political issues that may come up in future elections. They need to be removed as required summer reading ...." The administrator offered to help the teachers choose an alternate book.

Apparently, some books are simply too dangerous to place in the hands of even the bright students who populate AP courses.

Gene V Glass
San José State University
Arizona State University
University of Colorado Boulder
National Education Policy Center

The opinions expressed here are those of the author and do not represent the official position of San José State University, the National Education Policy Center, Arizona State University, or the University of Colorado Boulder.

Tuesday, February 27, 2018

Work, Play and the Loss of Relevance

William Doyle is a Rockefeller Resident Fellow and a Fulbright Scholar who spent time recently at the University of Eastern Finland. When he returned to New York City, he went in search of a Finnish-style public school and found it. It is called The Earth School. The experience started him thinking about things like work and play and schools and what education should be about. He wrote me a letter to which I have cobbled together a reply below.

First, Bill's letter:

Dear Gene,
I am enjoying reading about your fascinating, essential work in education. I am co-authoring a book with Pasi Sahlberg on the need for more play in school from pre-K through age 17. We are defining “play” as regular periods of freedom, choice and intellectual and physical play, both structured and unstructured, and both indoors and outdoors.

May I ask you 3 questions for our book?

  1. What is the importance of play in childhood education?
  2. How would you characterize the current state of play in school (pre-K through age 17)?
  3. What specific recommendations might you have for integrating play, both indoors and outdoors, into the "schools of tomorrow"?
Thank you very much for considering this!

With appreciation, Bill

Bill continued:
“My child now goes to PS 364, also known as the Earth School, a little-known gem of a public K-5 elementary in the East Village.
“The student population is some 50% black and Latino children. Half the students qualify for free and reduced priced lunch, and 23% of students receive special education services.
“If American teachers built a school, instead of politicians and bureaucrats, it would look a lot like this. Founded as an experimental program in 1992 by a group of New York City teachers who wanted, in the words of the school’s website, “to create a peaceful, nurturing place to stimulate learning in all realms of child development, intellectual, social, emotional and physical,” the Earth School is guided by the values of “hands-on exploration, an arts-rich curriculum, responsible stewardship of the Earth’s resources, harmonious resolution of conflict and parent-teacher partnership.
“While 'working rigorously in literacy and math' the students are encouraged “to explore, experiment, and even sometimes make a mess in the pursuit of learning.”
“The atmosphere of the school is one of warmth and safety. Teacher experience is prized here — the principal, Abbe Futterman, was one of the founding teachers of the school a quarter-century ago, and many other staff members have worked here for at least five or 10 years.
“Children at the school are assessed every day, not primarily by standardized tests — the majority of parents opt their kids out of state exams — but by certified, professional childhood educators who provide the ultimate in “personalized instruction”: the flesh-and-blood kind.
“Children at the school learn in part through play in the early years. They are encouraged to ask challenging questions and think for themselves. They are encouraged toto be creative and compassionate, and to make their own decisions. Children get unstructured, free-play outdoor recess in the big play yard most days.
“Like employees at Google who are given 20% of their time to devote to projects of their own choice, children are given a free afternoon every week to pursue their own self-chosen 'passion projects.'
“In a striking innovation I especially appreciate, parents are actually invited into the school and directly into the classrooms for the morning drop-off, and given a room in the heart of the schoo, to relax, chat and plan much-needed school fundraisers.
“The school is not perfect, and it is not for everybody. If you’re looking for universal iPads, data walls, digital learning badges or boot-camp behavior modification in your child’s classroom, you won’t find them here.
“But somehow, this oasis of child-centered, evidence-based childhood education has managed to survive and flourish for a quarter-century in the heart of the New York City public school system.
"If it can happen in New York City, it can happen everywhere. If we ever get over our love affair with testing, anything is possible. Even a normal childhood.
Bill's experiences gave me an opportunity to unburden myself of several grievances that have been brewing inside for some time.

First, the false opposites of WORK – PLAY confound thinking. Indeed, the puritanical attitude toward play is widespread, to the detriment not only of schooling but to all of life. If work aims to build, create, solve, or contribute, play cannot be any of these things. At best it is rest, necessary for future bouts of work. Virtually everything is wrong with this way of thinking. And it is a shame that it permeates all levels of the education system.

Let me focus on two things that are wrong: 1) play can lead to development of many kinds; 2) work is rapidly disappearing.

The idea that play is a state of being in which serious cognitive processing are turned off for the duration is simple nonsense. Such an idea harkens back to naïve thinking of fifty years ago about “culturally deprived” [minority] children. Persons who should have known better conceived of urban minority children existing in a state of suspended animation in which their minds were completely unengaged because of a lack of “inputs.” This was and is absurd, of course. When a person appears to be unconnected in a cognitive and conative way, it is simply that the observer is ignorant of what is going on in that person’s mind. That observers cannot conceive of what is going on in the mind of a child who is “at play,” does not mean that that child’s mind is not involved in complex and indelible learnings.

My colleague at Arizona State, James Gee, has spent a good deal of effort studying [working on] how children’s thinking is shaped by playing video games. If you haven’t seen his work, take a look at his Wikipedia page.

Personally, I see a lot of problems with video games. It’s not that children don’t learn a host of important things while playing them; it’s that I have real reservations about turning children over to the control of game developers – whose goals are basically commercial. Plus, I’m seeing too many of my own grandchildren opting for the screen in their bedroom rather than their age-mates outside the home. There are things to learn there too.

Secondly, the prejudice against play is shot through the education system. And it is just getting worse. Increasingly children are viewed as future employees of corporations who must be trained – at public expense – to contribute to the corporation’s bottom line. Standards and goals and tests are the instruments of production of the workforce of tomorrow, and they have infiltrated all the way down to kindergarten. Academics spin out 1,500 math objectives guaranteed to produce a high school graduate who can code for Facebook.

There are several things wrong with what has happened to our schools in relation to the WORK – PLAY dichotomy.

Why should the public pay for the training of workers for the corporations? The threat that untrained high school graduates will be unemployable is ridiculous. The vast majority of jobs involve brief training that is so particular to a time and place that it makes no sense to attempt it to a broad population of school children. Furthermore, the corporations’ wish to push job training off onto public schools is just another recessive tax passed to the general public. The corporations benefit from trained workers; they should train them themselves.

But worse than conceiving of all education as merely job/career education – as is increasingly happening – is simply closing one’s eyes to the inevitable disappearance of work. It is almost impossible to get people of all sorts to conceive of a world without work, and yet, everything is headed to exactly that end. Robotics and artificial intelligence are destroying jobs. Machines and electronic processors are replacing human beings who engage in work. Even Facebook’s Zuckerberg recently opined that tens of millions of jobs will become obsolete in a few decades. People who hear this find it incomprehensible. They say that there will be jobs for people to code the processors and build the machines. Yes, one person in ten thousand will do such work. But what about the other 9,999? In the full scope of human history, the time segment in which people thought of themselves as "having a career" will appear to be the mere blink of an eye. For eons passed, people hunted, fished or grew food. In a future that is rushing toward us, people's challenge will be how to put together a healthy and meaningful life in which machines do all the work.

Education is surely the most conservative institution in all of society — ignoring religion for the moment. It’s job is to transmit the culture to each new generation. Preservation of the culture is the very purpose of Burkean conservatism. Trying to get education to adapt to the reality of a world without work is like trying to relocate a cemetery. Instead of producing graduates who are job/career ready, educators should be asking themselves, “How do we prepare persons for a world without work?” I’m convinced that public education in all the industrialized nations is incapable of meeting this challenge. They are trapped in their conception of what the world will be and what there responsibilities are to prepare children for it. Moreover, should educators attempt to escape from the intellectual straight-jacket of “career-ready” education, they will be slapped down by conservative, corporate powers that control public schooling.

Instead of math – 90% antiquated and useless – and writing – soon to be completely replaced by recorded speech – and spelling – Autocorrect is writing as I type – and most of the remainder of the K-12 curriculum, we have to teach children how to engage their creative, playful, and emotional life. Learning about art and music are goals worthy of human beings freed from the drudgery of work. A liberal education was said to be education for a man (now person) freed – liberated – from work. A child should be helped to develop habits of mind and behavior that prevent their abusing their own bodies. Life-style sicknesses (diabetes, heart disease, various addictions) are epidemic and are one of the biggest detriments to the U.S. economy. Perhaps it is no coincidence that this has happened at the very same time that education has turned away from play.

Well, Bill. I suspect you can see where I am going with this. I won’t belabor all the details.

Gene V Glass
San José State University
Arizona State University
University of Colorado Boulder
National Education Policy Center

The opinions expressed here are those of the author and do not represent the official position of San José State University, the National Education Policy Center, Arizona State University, or the University of Colorado Boulder.

Tuesday, January 30, 2018

Arizona Ground Zero in Koch Brothers Attack on Public Schools

The following is quoted from the Washington Post article of January 30, 2018:

“In 2018, Koch donors see Arizona as ground zero in their push. Doug Ducey, the former chief executive of Cold Stone Creamery, became a member of the Koch network in 2011. Since 2015, he’s attended the seminars as governor of Arizona. Last year, he signed legislation to dramatically expand the state’s Empowerment Scholarship Accounts program so that students can use taxpayer dollars that would be spent on them in public schools to cover private-school tuition or other educational expenses.

“Teacher unions, worried that this will undermine the public system, collected enough signatures to put the law on hold and create a ballot proposition to let voters decide in November whether to expand vouchers.

“Addressing the seminar yesterday, Ducey touted the measure as further reaching than anything that’s been tried in other states. He warned that, under Arizona law, if advocates lose at the ballot box, they will not be able to legislate on the topic in the future. “This is a very real fight in my state,” Ducey said. “I didn’t run for governor to play small ball. I think this is an important idea.”

“The Koch network is likely to spend heavily to support the voucher law, setting up a battle royal with the labor movement.

“Ducey introduced Steve Perry, the headmaster of Capital Prep Charter Schools, who has been traveling Arizona to speak in support of the law. “The teacher unions are unencumbered by the truth,” he told the Koch donors. “It is a distant relative that is never invited to dinner.”

Gene V Glass
San José State University
Arizona State University
University of Colorado Boulder
National Education Policy Center

The opinions expressed here are those of the author and do not represent the official position of the National Education Policy Center, Arizona State University, University of Colorado Boulder, nor San José State University.

Wednesday, January 17, 2018

Senator Flake Comes of Age: An Open Letter

Dear Senator Jeff Flake:

Back in about 2000 when you were running for your seat in the Arizona House and you came to visit my colleague David Berliner and me in the Dean's Office at Arizona State University to lecture us on the infallibility of "market" control of public education, I wrote you off as a naive 38-year-old who would one day know better. Today, when you delivered your scathing and powerful denunciation of President Trump and his assault on truth and a free press, you finally came of age.

Your colleagues who condemn you and claim that you are only speaking out because you are not running in 2018 for a second term only draw attention to their own pusillanimity. Someday when this nightmare is over, you may run again; and I hope if you do, you will not forget how to speak truth to power.

Gene V Glass
Arizona State University
University of Colorado Boulder
National Education Policy Center
San José State University

The opinions expressed here are those of the author and do not represent the official position of the National Education Policy Center, Arizona State University, University of Colorado Boulder, nor San José State University.

Thursday, May 25, 2017

A Citizen's Encounter With a Charter School

I have on this blog exposed some of the unusual workings of Challenge charter school in Glendale, Arizona. In short, Greg Miller, who until recently, was Chairman of the Arizona State Board of Education. is the owner of Challenge charter school, from which he, his wife and his daughter draw more than $400,000 in salary annually. While this would set off conflict-of-interest alarms in nearly any other state in the Union, in Arizona it is simply business as usual. Last fall, the Arizona Governor decided he wanted to remove Miller from the State Board. Miller apparently could read the handwriting on the wall and he agreed to leave. As he walked out the door, Miller told the press that he agreed to quit if he got some assurances that the charter school he runs would get "political protections that I no longer could provide." He doesn't simply profit from his conflicts of interest, he's actually proud of them.

So when a parent recently contacted me about an incident of unprofessional treatment by the staff of Challenge charter school, I was not surprised. But that parent's experience should stand as a warning to other parents who consider doing business with the company known as Challenge Charter School: Arizona's First Official Core Knowledge School.

The parent speaks:

The other day we had a horrific experience with Wendy and Pam Miller [the principal and board member, and Greg Miller's daughter and wife, respectively] at Challenge Charter School. We had never met them before but we were the first on the list in 2015 for Kindergarten open-enrollment. We completed all subsequent paperwork on-time.

However, last Friday my wife received a patronizing voicemail from Wendy Miller stating that she reviewed our March 2017 updated information form and compared it to our February 2017 updated information form and found a discrepancy. She then continued by saying she was so sorry but that we would need to start from the beginning and complete an entirely new open-enrollment registration packet (if interested) and then be placed in the open-enrollment lottery. Moreover, she said “it’s not our policy to refund your $100 deposit but in this circumstance we will.” This was the least of our concerns. We were blind-sided by receiving this voicemail that had no information about the discrepancy or rationale, and it was the day before yesterday’s scheduled Kindergarten Assessment Boot Camp.

So, yesterday we showed up anyway and signed in at the table in the gym where Wendy Miller was standing. She looked at our daughters name and just stood there. I said “Hi, are you the principal?” and she said "yes, I am, let’s go to my office.” As we were walking from the gym to her office, I explained that we were blind-sided by her voicemail and that I would compare this to being accepted to college and then being told by the dean the day before that your enrollment is not being honored. She replied in a hostile and defensive manner “Don’t you dare be disrespectful to me like that do you understand? I am not going to have this.” So I remained silent and we proceeded to walk into the main office where she stopped at her mom’s office and said “I need back-up.” We proceeded to the conference room and they both basically belittled us over a question about if there was any special needs our child has received. On the February updated information sheet we said "No.” But on the March updated (Wendy called it refreshed form) information sheet we said “Yes, our daughter has received private OT speech therapy [which Wendy Miller in the attached video calls “significant” – we are not sure how she arrived at that considering how we simply wrote that she receives OT speech therapy.] The OT therapy we recently acquired for our daughter was (a) for speech enhancement (not that she has a speech problem) and (b) to help her not be frightened of loud noises (i.e., blender, public toilet flush, etc.). This was allowed by our health insurance so we accepted these services. We never requested special assessment because our daughter does not require it. She was already assessed by her therapist and any special needs therapy or placement was ruled out. But apparently, Wendy and Pam Miller jumped to conclusions and made this off-the-cuff assumption. However, they never initiated a prior dialogue with us about this before jumping to their conclusion which is apparently why they took away our daughter’s space.

They claimed that if there is a mismatch of information between February’s and March’s form then state law and their policy and procedures require that the parents lose their spot and have to complete new registration and be placed into a lottery. You will see from the attached video link that this is nonsense and that they have an ulterior motive (i.e., money/funding, etc). The fact of the matter is that the updated or refreshed information on the forms they require will not necessarily always match. That’s the point of asking a parent to provide updated information (i.e., address change, etc.) Therefore, there’s obviously something else going on here, and there seems to be a lot of "wrongs" happening. You will see in the attached video link (that I recorded on my cell phone) that the mother and daughter contradict one another too and not to mention are unprofessional school administrators (they have no checks and balances and there’s no fear of losing their job so they can behave like this).

One of the several contradictions you will notice in the video is when Wendy Miller says she doesn’t care what we wrote as the answer for the question at hand, but then her mother says that she’s already hired last month and essentially doesn’t have the resources for special testing (which, again we never asked for and isn’t necessary, but we weren’t given the opportunity to explain that). Additionally, Pam Miller claims that we weren’t truthful on the form. I then replied by asking her if we were lying and she said "Yes." Remember, this is a kindergarten. We didn’t steal from their office or commit a felony here. There are several other contradictions, aside from them shutting us down from having an intellectually honest conversation. Pam Miller says that this is an open-enrollment school and that no one has a spot until the first day of school. Well, we received a congratulatory letter in the mail saying our daughter’s enrollment had been accepted. Her statement is nonsense because parents have to know if their child has a spot well before the first day so that they can then find other options if denied. You will see additional contradictions, and will see that Wendy Miller tells us that this conversation is over.

I also have an 18-year-old boy (just graduated high school) and a 13-year-old girl (graduating 7th grade next week) in the Deer Valley Unified School District, and I’ve never experienced a crony capitalistic unprofessional school administration, like Challenge Charter School, up until now. It’s actually quiet shocking that this is allowed to continue without third-party inquiry into their inner-workings, especially since they are publically funded.

Video Link:

Shocking and regrettable! And an everyday occurrence in the charter school industry?
Gene V Glass
Arizona State University
University of Colorado Boulder
National Education Policy Center
San José State University

The opinions expressed here are those of the author and do not represent the official position of the National Education Policy Center, Arizona State University, University of Colorado Boulder, nor San José State University.

Friday, April 21, 2017

Is Your Job Robot-Proof?

It doesn’t take much observation of the typical high school curriculum to see that most of what is being taught is a vestige of a long past era. Geometry was essential to the ancient Egyptians, but not to modern Americans. Spelling, handwriting, and even writing itself have little to do with contemporary communication. My wife can’t understand why I still type instant messages instead of speak them into my smart phone.

The future is robots and artificial intelligence. Profit-seeking individuals and corporations demand it. Ford motor company recently ear-marked $1 billion for R&D on robotics, in an industry in which humans are already disappearing from the assembly line.

My father left school at age 14 and started work as an apprentice printer in 1923. He worked as a dues-paying member of the International Typographical Union for nearly 40 years. In the mid-1960s, his union struck against the introduction of computers into the composing room. He never worked another day as a printer. When he retired, the ITU’s pension fund was broke. "Printer" was the first occupation to be wiped out by computers. But it was hardly the last.

Recently, David Brancaccio and Katy Long undertook to catalogue various occupations as either 100% robot-proof (i.e., unlikely to be replaced by computers and AI) or 0% robot-proof. My father’s job was 0%, and it happened so long ago that Brancaccio and Long didn’t even bother to mention it. Here are their two lists:

    0% Automatable (Most Robot-Proof)
  • Ambulance Drivers and Attendants, Except Emergency Medical Technicians
  • Animal Scientists
  • Animal Trainers
  • Astronomers
  • Athletes and Sports Competitors
  • Clergy
  • Dancers
  • Directors, Religious Activities and Education
  • Historians
  • Mathematical Technicians
  • Models
  • Music Directors and Composers
  • Religious Workers, All Other
  • Roof Bolters, Mining
    100% Automatable (Least Robot-Proof)
  • Aircraft Cargo Handling Supervisors
  • Dredge Operators
  • Foundry Mold and Coremakers
  • Graders and Sorters, Agricultural Products
  • Logging Equipment Operators
  • Machine Feeders and Offbearers
  • Medical Appliance Technicians
  • Motion Picture Projectionists
  • Ophthalmic Laboratory Technicians
  • Packaging and Filling Machine Operators and Tenders
  • Plasterers and Stucco Masons
  • Slaughterers and Meat Packers
We can conclude a couple of things from these lists.

1) The robot-proof jobs have to do with the arts, sports, entertainment, and – shall we say – spiritual pursuits.

2) The jobs replaceable by computers, AI, and robots are the mid-level trades that employ the bulk of the nation’s workforce.

Going beyond the lists to the question of what is the role of public education in job training for the future, one can only conclude that our schools are in big trouble – and not in the way that most people think of trouble. Most of what is being taught is worthless, either for personal development or for life as a wage earner after schooling is done. Virtually all of what is tested for in the current madness of high-stakes paper-and-pencil achievement testing is irrelevant. It won’t prepare you for a job, and it won’t enrich your life for all those hours, days, weeks and years ahead when you are not working.

The transformation of work that is going on all around us is of utmost importance. It will have major implications for a topic that that absolutely paralyzing the thinking of political conservatives everywhere: entitlements. What will become of tens of millions of people who have no way to contribute to the nation’s economy? Will that tiny fraction of the population who can create real value support them, or will they look down their noses at them and ignore them? Few are willing to face the implications of a future of no work without moralizing or yearning for an atavistic era that will never return.

Is your job robot-proof? The answer for the vast majority of the U.S. workforce is regrettably, "No."

Gene V Glass
Arizona State University
University of Colorado Boulder
National Education Policy Center
San José State University

The opinions expressed here are those of the author and do not represent the official position of the National Education Policy Center, Arizona State University, University of Colorado Boulder, nor San José State University.

Friday, April 14, 2017

What Goes Around Comes Around: Voucher Scammers Get Scammed

Republicans in the Arizona legislature recently passed a significant expansion of the "Empowerment Scholarship" program -- a thinly disguised voucher program adopted years ago and slightly expanded in each subsequent year.

Originally intended only for special needs students, it was broadened to include children of military serving in Iraq & Afghanistan, and then children living on Indian reservations. The cynical intent is obvious.

The latest incarnation of the program will expand the program by 5,000 students per year until a cap of 30,000 is reached.

Even Republicans were reluctant to support the expansion, probably because of persistent non-support of vouchers among the voting public. The latest PDK Gallup poll continues to show more than 60% of parents opposed.

Big lobby pressure to expand the program came from the local Goldwater Institute. When a compromise on the 5,000 per year expansion was reached, the reluctant Republicans fell in line.

But as soon as the bill was signed by AZ governor Ducey, Goldwater CEO Darcy Olsen sent emails to the Institute's donors and friends stating that soon they would achieve a lifting of all limits. Republicans in the legislature felt betrayed, as they obviously were.

So deception, mendacity, and treachery are the order of the day in the nation's #1 legislature for school choice.

Gene V Glass
Arizona State University
University of Colorado Boulder
National Education Policy Center
San José State University

The opinions expressed here are those of the author and do not represent the official position of the National Education Policy Center, Arizona State University, University of Colorado Boulder, nor San José State University.

Friday, April 7, 2017

Arizona's Neo-Vouchers: The Camel is in the Tent

NEPC's Kevin Welner called them "neo-vouchers": tax credits and "scholarships" that attempt to hide their intent behind deceitful labels.

On Thursday night, April 6, 2017, a Republican dominated Arizona Legislature passed a significant expansion of a voucher program that has been in effect since 2011. Governor Ducey, founder of the ice cream parlor chain Cold Stone Creamery, promised to sign the bill into law.

The Empowerment Scholarships -- vouchers by another name -- were originally available only to a highly limited number of students: those with special needs; children of military personnel stationed in areas of conflict, and a few others. But in classic camel's-nose-under-the-tent fashion, each year the Legislature pushed the limits a little broader: students on Indian reservations, for example. The "scholarships" may be redeemed at private schools, religious or otherwise. Recent research has established that the vouchers are going primarily to upper-middle class families.

The bill that will soon become Arizona law will gradually expand the program to all sorts of students until a cap of 30,000 is reached in a few years. But expect the boundaries to expand further as an emboldened Legislature introduces future bills.

With a sizable portion of its students being Hispanic and a sizable portion of its taxpayers being White retirees, look to Arizona to be the leader in the destruction of the public school system.

Gene V Glass
Arizona State University
University of Colorado Boulder
National Education Policy Center
San José State University

The opinions expressed here are those of the author and do not represent the official position of the National Education Policy Center, Arizona State University, University of Colorado Boulder, nor San José State University.

Friday, February 17, 2017

Fed's "Gold Standard" Produces Fool's Gold

As a new administration moves into the US Department of Education, the opportunity arises to review and assess the Department's past practices. A recent publication goes to the heart of how US DOE has been attempting to influence public education. Unfortunately, in an effort to justify millions of dollars spent on research and development, bureaucrats pushed a favorite instructional program that teachers flatly rejected.

The Gold Standard

There is a widespread belief that the best way to improve education is to get practitioners to adopt practices that "scientific" methods have proven to be effective. These increasingly sophisticated methods are required by top research journals and for federal government improvement initiatives such as Investing in Innovation (i3) Initiative to fund further research or dissemination efforts. The US DOE established the What Works Clearinghouse (WWC) to identify the scientific gold-standards and apply them to certify for practitioners which programs "work." The Fed's "gold standard" is the Randomized Comparative Trial (RCT). In addition, there have been periodic implementations of US DOE policies that require practitioners to use government funds only for practices that the US DOE has certified to be effective.

However, an important new article published by Education Policy Analysis Archives, concludes that these gold-standard methods misrepresent the actual effectiveness of interventions and thereby mislead practitioners by advocating or requiring their use. The article is entitled “The Failure of the U.S. Education Research Establishment to Identify Effective Practices: Beware Effective Practices Policies.”

The Fool's Gold

Earlier published work by the author, Professor Stanley Pogrow of San Francisco State University, found that the most research validated program, Success for All, was not actually effective. Quite the contrary! Pogrow goes further and analyzes why these gold-standard methods can not be relied on to guide educators to more effective practice.

The Need For a New Standard

Key problems with the Randomized Comparative Trial include (1) the RCT almost never tells you how the experimental students actually performed, (2) that the difference between groups that researchers use to consider a program to be effective is typically so small that it is “difficult to detect” in the real world, and (3) statistically manipulating the data to the point that the numbers that are being compared are mathematical abstractions that have no real world meaning—and then trying to make them intelligible with hypothetical extrapolations such the difference favoring the experimental students is the equivalent of increasing results from the 50th to the 58th percentile, or an additional month of learning. The problem is that we do not know if the experimental students actually scored at the 58th or 28th percentile. So in the end, we end up not knowing how students in the intervention actually performed, and any benefits that are found are highly exaggerated.

Pogrow also shows that the notion that science requires the use of RCTs is wrong. Even the medical profession, which does use gold-standard experimental techniques to test the effectiveness of medicines, uses much simpler scientific methods without controlled experiments in other areas of clinical practice such as obstetrics and improving health care delivery in complex organizations such as hospitals. The latter methods, called “improvement science,” appear more relevant to identifying scalable effective practices in the complex settings of schools.

The “rigorous” gold-standard WWC-type method derived from the traditions of the psychology lab are also creating misleading results for clinical practice in psychology and psychiatry. There is increasing criticism of the practicality and applicability of the RCT for guiding professional practice in all complex organizations.

Dr. Pogrow concludes that current federal initiatives to disseminate programs “proven to be effective” are of little value, and that a recent call to establish a new round of expanded “Effective Practices Policies” is a dangerous and unwarranted intrusions into local decision-making. He urges the profession to resist seductive calls and bureaucratic pressure from the US DOE to adopt such policies. His message for the Feds and his colleagues in the professoriate is to (1) suspend the activities of the What Works Clearinghouse and i3 dissemination efforts, and to (2) develop methods that can certify which interventions consistently produce clearly observable benefits in real world context. The latter requires an expanded conception of what science is and how we teach applied research methods.

Gene V Glass
Arizona State University
University of Colorado Boulder
National Education Policy Center
San José State University

The opinions expressed here are those of the author and do not represent the official position of the National Education Policy Center, Arizona State University, University of Colorado Boulder, nor San José State University.