Monday, January 30, 2012

Who's at Fault Here?

The town of San Luis sits on the Arizona border in the southwest corner of the state. But 25,000 of the inhabitants live on the Arizona side of the border and 175,000 live on the Mexico side. So when a young woman who had graduated from high school on the Arizona side of San Luis filed as a candidate for city council, someone lodged the complaint that if elected she would violate the state law that says that anyone holding political office must be proficient in English. The judge who ajudicated the case ruled that the young woman's name must be stricken from the ballot because her command of English was not sufficient. The issues involved here seem obvious. The prevailing language in San Luis is Spanish. The Arizona law makes no attempt to define "proficiency" in English.

Who is at fault here? The high school? The State of Arizona and its laws? The judge?

Gene V Glass
University of Colorado Boulder
Arizona State University

Saturday, January 28, 2012

The Problem with Value-Added Measures

Matthew Di Carlo in his blog on January 26th wrote this in a discussion of Florida's use of value-added assessment of teachers and schools:
I would argue that growth-based measures are the only ones that, if they’re designed and interpreted correctly, actually measure the performance of the school or district in any meaningful way....
Those italics are in the original, and they are a bit of a cop-out. In my opinion there is no way to "design and interpret correctly" the various growth measures that have been proposed for the measurement of the contribution of a teacher or even a group of teachers to a group of children's learning. In the first place, any system of high stakes, punitive measurement of teachers for purposes of monetary rewards or other benefits produces not just teaching to the test—a problem so pervasive that even the President of the U.S. can talk about in a State of the Union address—but also produces cheating...and before those feelings of moral outrage begin to take you over, please summon the honesty to admit that you would too if placed in the same circumstance.

But there is another problem with the value-added measures that is much too infrequently talked about.

Way back in the 1990s while moderating an online discussion of the Tennessee Value-Added Assessment System (due to a business prof at U Tenn named William Sanders, I believe), Sanders himself, and later his assistant and occasional co-author, Sandra Horn made a brief appearance in the discussion and quickly retreated. That discussion among a dozen or more scholars runs to several thousand words and is available to anyone here: http://gvglass.info/TVAAS/.

I happened to be giving a talk in Denver in the mid-1990s at a conference in Denver where Sanders was also speaking. After my talk I was approached by a young woman who identified herself as Horn and asked if I had time for a brief conversation. "Yes, certainly." Horn started off by saying that Sanders and she felt that if I just understood a few things about TVAAS that the objections I had expressed in the online discussion would surely be cleared up. "Try me."

For 15 minutes I listened to descriptions of TVAAS that were entirely irrelevant to my objections. Finally I interrupted:

GVG: Let me pose a hypothetical to you. Suppose that there are two classes of children and that Class A and Class B are taught by two teachers who teach in exactly the same way. In fact, every word, action, and thought they produce is identical. And suppose further that these two groups of children begin the school year with identical knowledge acquired in the past. Now here is the critical assumption. Suppose that the pupils in Class A have an average IQ of 75, and the pupils in Class B have an average IQ of 125. Do you believe that your measure of teacher value-added will produce the same numeric value for these two teachers?

SH: Yes.

Rather than deliver an impromptu lecture on the difference between aptitude (mental ability, a portion of which is undeniably inherited) and school achievement, I excused myself.

And such is the Achilles heel in all of the so-called value-added assessment systems. They act as though the statistical equating on achievement tests (as fallible as it is) of groups of students has held all influences constant (ceteris paribus), and hence the gain score is valid and fair as a measure of the contribution to learning of a teacher or a school. It is not, and never will be.

Gene V Glass
University of Colorado Boulder
Arizona State University

Friday, January 27, 2012

Obama: "Keep Your Free Lunches Out of Our Schools"

A bill just passed the Arizona House that permits ("encourages") school districts to drop out of the federal free and reduced price lunch program for poor children. ("The National School Lunch Program (NSLP) is a federally assisted meal program operating in public schools.... It provides nutritionally balanced, low-cost or free lunches to children .... The program was established under the National School Lunch Act, signed by President Harry Truman in 1946.")

This would appear to be simply another instance of Arizona Conservatives (the House is overwhelmingly Republican) telling the hated federal government to stay out of Arizona's business. But on closer examination, it could also be about "business" itself. The trend toward commercial interests—soft-drink, fast food, athletic shoe and clothing companies—seeking access to the captive market public school children is well documented. (See the annual report by Molnar, Boninger & Fogarty.) Running federally subsidized or provided lunches out of the schools would expand the market for these companies.

Lobbyists feed legislators; Pepsi and Pizza Hut feed children; and everyone gets fat.

Gene V Glass
University of Colorado Boulder
Arizona State University

Tuesday, January 24, 2012

Ranking Education Policy Scholars

Rick Hess recently (January 2012) published the second of what is promised to be an annual ranking of university-based scholars who influence the nation’s education policy. (See http://tinyurl.com/8x4f34z.) His intent in publishing the rankings is to “… recognize those university-based academics who are contributing most substantially to public debates about schools and schooling.“ His ranking of more than 100 persons is derived from a combination of seven variables: Google Scholar citations, “Book Points,” Highest Amazon Ranking, Education Press mentions, Education Blog mentions, Newspaper mentions, and Congressional Record mentions. Rick and his assistants are to be thanked not only for opening a discussion of scholarly influence on education policy but for making available to the public the data which were gathered at considerable expense in time and money.

Casual inspection of the matrix of intercorrelations of these seven variables suggests to an old psychometrician that the complex is lumpy. The first three intercorrelate almost .5, the second three intercorrelate almost .7 and the cross correlations among these two groups average less than .3. Accordingly, I performed a factor analysis of the seven variables. It revealed two factors, one defined by the first three variables , the other by the second three. Let’s name them Academic Presence and Media Presence.

Hess’s 121 individuals can be scored and ranked on each of these two factors. The complete rankings along with all the data can be seen here: http://gvglass.info/papers/edpolicy-people.pdf. In the interest of saving space, let’s focus on just the top 50 on each factor.

The Ranking on Academic Presence

1

Larry Cuban

Stanford

40.3602

2

Linda Darling-Hammond

Stanford

37.0284

3

Nel Noddings

Stanford

32.7099

4

Diane Ravitch

NYU

31.1064

5

Paul Peterson

Harvard

31.0503

6

Eric Hanushek

Stanford

28.2065

7

Terry Moe

Stanford

25.5173

8

Carol Tomlinson

U Virginia

25.0082

9

Gary Orfield

UCLA

24.0278

10

Gloria Ladson-Billings

U Wisconsin

23.1723

11

Richard Elmore

Harvard

23.0128

12

Henry Levin

Columbia (TC)

22.0085

13

David K. Cohen

U Michigan

21.5601

14

David Berliner

Arizona State U

21.2681

15

Daniel Koretz

Harvard

20.8576

16

James Comer

Yale

19.9633

17

Pedro Noguera

NYU

19.7797

18

Gene Glass

U Colorado Boulder

19.7384

19

Kenneth Zeichner

U Washington

19.283

20

Marcelo Suarez-Orozco

NYU

18.7308

21

Camilla Benbow

Vanderbilt

18.1655

22

Sonia Nieto

U Mass-Amherst

17.7648

23

Caroline Hoxby

Stanford

17.6055

24

Robert Pianta

U Virginia

17.1553

25

Bruce Fuller

UC Berkeley

16.9575

26

Marilyn Cochran-Smith

Boston College

15.8067

27

Richard Arum

NYU

14.8000

28

Richard Murnane

Harvard

14.2628

29

Daniel Willingham

U Virginia

14.0000

30

Andrew Porter

U Pennsylvania

13.7939

31

Susan Fuhrman

Columbia (TC)

13.4245

32

Deborah Ball

U Michigan

12.3928

33

Jay Greene

U Arkansas

12.3246

34

Kenneth Wong

Brown

12.3189

35

Anthony Bryk

Stanford

12.1063

36

Michael W. Kirst

Stanford

11.8413

37

Jeffrey Henig

Columbia (TC)

11.8195

38

James W. Guthrie

SMU

11.7428

39

David Labaree

Stanford

11.3155

40

David Breneman

U Virginia

11.1250

41

Thomas J. Kane

Harvard

11.0819

42

Julian R. Betts

UC San Diego

10.8709

43

Lorrie Shepard

U Colorado Boulder

10.7108

44

Paul T. Hill

U Washington

10.6569

45

Adam Gamoran

U Wisconsin

10.5953

46

Roland Fryer

Harvard

10.3190

47

Amy Stuart Wells

Columbia (TC)

10.1929

48

Douglas Harris

U Wisconsin

10.1700

49

Susan Moore Johnson

Harvard

10.1525

50

Sara Goldrick-Rab

U Virginia

10.0730


The Ranking on Media Presence

1

Diane Ravitch

NYU

23.2778

2

Richard Arum

NYU

21.6879

3

Linda Darling-Hammond

Stanford

16.9698

4

Dan Goldhaber

U Washington

8.8060

5

Robert Pianta

U Virginia

8.5612

6

Adam Gamoran

U Wisconsin

8.2492

7

Eric Hanushek

Stanford

7.4840

8

Anthony Bryk

Stanford

7.0636

9

Thomas J. Kane

Harvard

6.9967

10

Deborah Ball

U Michigan

6.5772

11

Douglas Staiger

Dartmouth

6.0465

12

Eric Bettinger

Stanford

5.5942

13

Jal Mehta

Harvard

5.3505

14

David Figlio

Northwestern

5.1720

15

Robin J. Lake

U Washington

4.9470

16

Bruce D. Baker

Rutgers

4.9275

17

Roland Fryer

Harvard

4.9200

18

Donald Heller

Penn State

4.9142

19

Bridget Terry Long

Harvard

4.8177

20

Susanna Loeb

Stanford

4.7537

21

Lorrie Shepard

U Colorado Boulder

4.5772

22

Michael W. Kirst

Stanford

4.5692

23

Richard Elmore

Harvard

4.5663

24

David Berliner

Arizona State U

4.3562

25

Richard Ingersoll

U Indiana

4.3372

26

Bruce Fuller

UC Berkeley

4.2795

27

Marilyn Cochran-Smith

Boston College

4.1884

28

Gary Miron

Western Michigan U

4.1505

29

Patrick J. Wolf

U Arkansas

3.9820

30

Paul T. Hill

U Washington

3.8965

31

Jonathan Plucker

Indiana U

3.8201

32

John H. Tyler

Brown

3.7072

33

Kris Gutierrez

U Colorado Boulder

3.6685

34

Michael Podgursky

U Missouri

3.6662

35

Thomas Dee

U Virginia

3.5330

36

Kenneth Zeichner

U Washington

3.4194

37

Dominic Brewer

USC

3.2112

38

Sarah E. Turner

U Virginia

3.1892

39

Michael Feuer

George Washington U

3.1205

40

Ronald Ferguson

Harvard

3.0053

41

Carol Lee

Northwestern

2.8680

42

Eva Baker

UCLA

2.5725

43

Richard Murnane

Harvard

2.5580

44

Jay Greene

U Arkansas

2.5486

45

Julian R. Betts

UC San Diego

2.4670

46

Heather C. Hill

Harvard

2.4367

47

Catherine Lugg

Rutgers

2.3822

48

Priscilla Wohlstetter

USC

2.2850

49

Jacob Vigor

Duke

2.2830

50

Dale Ballou

Vanderbilt

2.2017

The geographic distribution of the institutions of the 121 scholars is interesting to observe. The map below shows the locations of those universities with 2 or more persons in the list of 121. You can see a larger version of the map in the full report at http://gvglass.info/papers/edpolicy-people.pdf.
Also in the full report is a listing of the 17 scholars who scored in the top 50 ranks on each dimension.

Gene V Glass
University of Colorado Boulder
Arizona State University