Who's at Fault Here?

The town of San Luis sits on the Arizona border in the southwest corner of the state. But 25,000 of the inhabitants live on the Arizona side of the border and 175,000 live on the Mexico side. So when a young woman who had graduated from high school on the Arizona side of San Luis filed as a candidate for city council, someone lodged the complaint that if elected she would violate the state law that says that anyone holding political office must be proficient in English. The judge who ajudicated the case ruled that the young woman's name must be stricken from the ballot because her command of English was not sufficient. The issues involved here seem obvious. The prevailing language in San Luis is Spanish. The Arizona law makes no attempt to define "proficiency" in English.

Who is at fault here? The high school? The State of Arizona and its laws? The judge?

Gene V Glass
University of Colorado Boulder
Arizona State University

The Problem with Value-Added Measures

Matthew Di Carlo in his blog on January 26th wrote this in a discussion of Florida's use of value-added assessment of teachers and schools:

I would argue that growth-based measures are the only ones that, if they’re designed and interpreted correctly, actually measure the performance of the school or district in any meaningful way....

Those italics are in the original, and they are a bit of a cop-out. In my opinion there is no way to "design and interpret correctly" the various growth measures that have been proposed for the measurement of the contribution of a teacher or even a group of teachers to a group of children's learning. In the first place, any system of high stakes, punitive measurement of teachers for purposes of monetary rewards or other benefits produces not just teaching to the test—a problem so pervasive that even the President of the U.S. can talk about in a State of the Union address—but also produces cheating...and before those feelings of moral outrage begin to take you over, please summon the honesty to admit that you would too if placed in the same circumstance.

But there is another problem with the value-added measures that is much too infrequently talked about.

Way back in the 1990s while moderating an online discussion of the Tennessee Value-Added Assessment System (due to a business prof at U Tenn named William Sanders, I believe), Sanders himself, and later his assistant and occasional co-author, Sandra Horn made a brief appearance in the discussion and quickly retreated. That discussion among a dozen or more scholars runs to several thousand words and is available to anyone here: http://gvglass.info/TVAAS/.

I happened to be giving a talk in Denver in the mid-1990s at a conference in Denver where Sanders was also speaking. After my talk I was approached by a young woman who identified herself as Horn and asked if I had time for a brief conversation. "Yes, certainly." Horn started off by saying that Sanders and she felt that if I just understood a few things about TVAAS that the objections I had expressed in the online discussion would surely be cleared up. "Try me."

For 15 minutes I listened to descriptions of TVAAS that were entirely irrelevant to my objections. Finally I interrupted:

GVG: Let me pose a hypothetical to you. Suppose that there are two classes of children and that Class A and Class B are taught by two teachers who teach in exactly the same way. In fact, every word, action, and thought they produce is identical. And suppose further that these two groups of children begin the school year with identical knowledge acquired in the past. Now here is the critical assumption. Suppose that the pupils in Class A have an average IQ of 75, and the pupils in Class B have an average IQ of 125. Do you believe that your measure of teacher value-added will produce the same numeric value for these two teachers?

SH: Yes.

Rather than deliver an impromptu lecture on the difference between aptitude (mental ability, a portion of which is undeniably inherited) and school achievement, I excused myself.

And such is the Achilles heel in all of the so-called value-added assessment systems. They act as though the statistical equating on achievement tests (as fallible as it is) of groups of students has held all influences constant (ceteris paribus), and hence the gain score is valid and fair as a measure of the contribution to learning of a teacher or a school. It is not, and never will be.

Gene V Glass
University of Colorado Boulder
Arizona State University

Obama: "Keep Your Free Lunches Out of Our Schools"

A bill just passed the Arizona House that permits ("encourages") school districts to drop out of the federal free and reduced price lunch program for poor children. ("The National School Lunch Program (NSLP) is a federally assisted meal program operating in public schools.... It provides nutritionally balanced, low-cost or free lunches to children .... The program was established under the National School Lunch Act, signed by President Harry Truman in 1946.")

This would appear to be simply another instance of Arizona Conservatives (the House is overwhelmingly Republican) telling the hated federal government to stay out of Arizona's business. But on closer examination, it could also be about "business" itself. The trend toward commercial interests—soft-drink, fast food, athletic shoe and clothing companies—seeking access to the captive market public school children is well documented. (See the annual report by Molnar, Boninger & Fogarty.) Running federally subsidized or provided lunches out of the schools would expand the market for these companies.

Lobbyists feed legislators; Pepsi and Pizza Hut feed children; and everyone gets fat.

Gene V Glass
University of Colorado Boulder
Arizona State University

Ranking Education Policy Scholars

Rick Hess recently (January 2012) published the second of what is promised to be an annual ranking of university-based scholars who influence the nation’s education policy. (See http://tinyurl.com/8x4f34z.) His intent in publishing the rankings is to “… recognize those university-based academics who are contributing most substantially to public debates about schools and schooling.“ His ranking of more than 100 persons is derived from a combination of seven variables: Google Scholar citations, “Book Points,” Highest Amazon Ranking, Education Press mentions, Education Blog mentions, Newspaper mentions, and Congressional Record mentions. Rick and his assistants are to be thanked not only for opening a discussion of scholarly influence on education policy but for making available to the public the data which were gathered at considerable expense in time and money.

Casual inspection of the matrix of intercorrelations of these seven variables suggests to an old psychometrician that the complex is lumpy. The first three intercorrelate almost .5, the second three intercorrelate almost .7 and the cross correlations among these two groups average less than .3. Accordingly, I performed a factor analysis of the seven variables. It revealed two factors, one defined by the first three variables , the other by the second three. Let’s name them Academic Presence and Media Presence.

Hess’s 121 individuals can be scored and ranked on each of these two factors. The complete rankings along with all the data can be seen here: http://gvglass.info/papers/edpolicy-people.pdf. In the interest of saving space, let’s focus on just the top 50 on each factor.

The Ranking on Academic Presence

1	Larry Cuban	Stanford	40.3602
2	Linda Darling-Hammond	Stanford	37.0284
3	Nel Noddings	Stanford	32.7099
4	Diane Ravitch	NYU	31.1064
5	Paul Peterson	Harvard	31.0503
6	Eric Hanushek	Stanford	28.2065
7	Terry Moe	Stanford	25.5173
8	Carol Tomlinson	U Virginia	25.0082
9	Gary Orfield	UCLA	24.0278
10	Gloria Ladson-Billings	U Wisconsin	23.1723
11	Richard Elmore	Harvard	23.0128
12	Henry Levin	Columbia (TC)	22.0085
13	David K. Cohen	U Michigan	21.5601
14	David Berliner	Arizona State U	21.2681
15	Daniel Koretz	Harvard	20.8576
16	James Comer	Yale	19.9633
17	Pedro Noguera	NYU	19.7797
18	Gene Glass	U Colorado Boulder	19.7384
19	Kenneth Zeichner	U Washington	19.283
20	Marcelo Suarez-Orozco	NYU	18.7308
21	Camilla Benbow	Vanderbilt	18.1655
22	Sonia Nieto	U Mass-Amherst	17.7648
23	Caroline Hoxby	Stanford	17.6055
24	Robert Pianta	U Virginia	17.1553
25	Bruce Fuller	UC Berkeley	16.9575
26	Marilyn Cochran-Smith	Boston College	15.8067
27	Richard Arum	NYU	14.8000
28	Richard Murnane	Harvard	14.2628
29	Daniel Willingham	U Virginia	14.0000
30	Andrew Porter	U Pennsylvania	13.7939
31	Susan Fuhrman	Columbia (TC)	13.4245
32	Deborah Ball	U Michigan	12.3928
33	Jay Greene	U Arkansas	12.3246
34	Kenneth Wong	Brown	12.3189
35	Anthony Bryk	Stanford	12.1063
36	Michael W. Kirst	Stanford	11.8413
37	Jeffrey Henig	Columbia (TC)	11.8195
38	James W. Guthrie	SMU	11.7428
39	David Labaree	Stanford	11.3155
40	David Breneman	U Virginia	11.1250
41	Thomas J. Kane	Harvard	11.0819
42	Julian R. Betts	UC San Diego	10.8709
43	Lorrie Shepard	U Colorado Boulder	10.7108
44	Paul T. Hill	U Washington	10.6569
45	Adam Gamoran	U Wisconsin	10.5953
46	Roland Fryer	Harvard	10.3190
47	Amy Stuart Wells	Columbia (TC)	10.1929
48	Douglas Harris	U Wisconsin	10.1700
49	Susan Moore Johnson	Harvard	10.1525
50	Sara Goldrick-Rab	U Virginia	10.0730

---

The Ranking on Media Presence

1	Diane Ravitch	NYU	23.2778
2	Richard Arum	NYU	21.6879
3	Linda Darling-Hammond	Stanford	16.9698
4	Dan Goldhaber	U Washington	8.8060
5	Robert Pianta	U Virginia	8.5612
6	Adam Gamoran	U Wisconsin	8.2492
7	Eric Hanushek	Stanford	7.4840
8	Anthony Bryk	Stanford	7.0636
9	Thomas J. Kane	Harvard	6.9967
10	Deborah Ball	U Michigan	6.5772
11	Douglas Staiger	Dartmouth	6.0465
12	Eric Bettinger	Stanford	5.5942
13	Jal Mehta	Harvard	5.3505
14	David Figlio	Northwestern	5.1720
15	Robin J. Lake	U Washington	4.9470
16	Bruce D. Baker	Rutgers	4.9275
17	Roland Fryer	Harvard	4.9200
18	Donald Heller	Penn State	4.9142
19	Bridget Terry Long	Harvard	4.8177
20	Susanna Loeb	Stanford	4.7537
21	Lorrie Shepard	U Colorado Boulder	4.5772
22	Michael W. Kirst	Stanford	4.5692
23	Richard Elmore	Harvard	4.5663
24	David Berliner	Arizona State U	4.3562
25	Richard Ingersoll	U Indiana	4.3372
26	Bruce Fuller	UC Berkeley	4.2795
27	Marilyn Cochran-Smith	Boston College	4.1884
28	Gary Miron	Western Michigan U	4.1505
29	Patrick J. Wolf	U Arkansas	3.9820
30	Paul T. Hill	U Washington	3.8965
31	Jonathan Plucker	Indiana U	3.8201
32	John H. Tyler	Brown	3.7072
33	Kris Gutierrez	U Colorado Boulder	3.6685
34	Michael Podgursky	U Missouri	3.6662
35	Thomas Dee	U Virginia	3.5330
36	Kenneth Zeichner	U Washington	3.4194
37	Dominic Brewer	USC	3.2112
38	Sarah E. Turner	U Virginia	3.1892
39	Michael Feuer	George Washington U	3.1205
40	Ronald Ferguson	Harvard	3.0053
41	Carol Lee	Northwestern	2.8680
42	Eva Baker	UCLA	2.5725
43	Richard Murnane	Harvard	2.5580
44	Jay Greene	U Arkansas	2.5486
45	Julian R. Betts	UC San Diego	2.4670
46	Heather C. Hill	Harvard	2.4367
47	Catherine Lugg	Rutgers	2.3822
48	Priscilla Wohlstetter	USC	2.2850
49	Jacob Vigor	Duke	2.2830
50	Dale Ballou	Vanderbilt	2.2017

The geographic distribution of the institutions of the 121 scholars is interesting to observe. The map below shows the locations of those universities with 2 or more persons in the list of 121. You can see a larger version of the map in the full report at http://gvglass.info/papers/edpolicy-people.pdf.
Also in the full report is a listing of the 17 scholars who scored in the top 50 ranks on each dimension.

Gene V Glass
University of Colorado Boulder
Arizona State University