The Syndicate's Hands

Someone in San Diego, who used Tony Alvarado as a search query, has been reading this blog. I hope he reads this:
This comes from the dailyhowler and it points to the deception in the Alvarado methodology, which is a lynchpin of Klein's.
A quick summary, maybe it works in some situations, but it's not going to work in Ocean Hill Brownsville's and the South Bronx.
http://www.dailyhowler.com/dh110905.shtml
Special report: Where are standards?
PART 3—NOT THAT EASY TO SAY: It’s a bit amazing to see the way Hedrick Smith treats District 2—to see how he presents the District 2 record in his PBS program, Making Schools Work. Smith spends about fifteen minutes considering the tenure of Anthony Alvarado, former District 2 school chief (see THE DAILY HOWLER, 11/8/05). And quite literally, just forty seconds into the segment, we see him praised, as follows:
SMITH (10/5/05): The man behind the reform in District 2 was Tony Alvarado. He’d been working in New York City schools for twenty years, was even chancellort briedfly, and then became supoerinetendent of District 2 in 1987.
RESNICK: Tony Alvarado was a turning point. The District 2 experiment was a turning point. It showed what could be done with serious, central leadership.
The testimonial comes from Lauren Resnick, identified only as “Lauren Resnick, University of Pittsburgh.” And at the end of the segment on District 2, Resnick pops up again for a closing statement. “Tony Alvarado took a stand on what good teaching is,” she says. “And he showed that when you did that you could make many schools bloom—not just a few.” Again, Resnick is IDed only as an academic. She’s “Lauren Resnick, University of Pittsburgh.” That’s all Smith’s viewers are told.
But Resnick is not a detached observer of the District 2 reforms. As we noted yesterday, Andrew Wolf complained in a recent column about the way Smith presents—and fails to identify—some of his subject’s key associates:
WOLF (10/7/05): Mr. Alvarado's critics here [in New York] were omitted from the program. Nor did Mr. Smith properly disclose that Elaine Fink, Mr. Alvarado's deputy and frequent defender during the program, is also Mrs. Alvarado. Nor did he point out that Lauren Resnick, another Alvarado booster, has received millions in staff development contracts resulting from her association with Mr. Alvarado.
We can’t vouch for Wolf’s dollar figure (nor do we doubt it), but Resnick is a close professional associate of Alvarado and Fink, as Smith clearly knows. (On this page, his web site links to this 1999 academic paper. Co-authored by Resnick and Fink, it praises the District 2 reforms.) Resnick is entitled to her opinions, of course, and it may be that her views are correct. But forty seconds into this segment, Smith lets Resnick cheerlead for Alvarado—and he fails to tell us of the connection. When you see this kind of “reporting,” you’re forced to ask: Readers! Where are standards?

No, Smith doesn’t present Alvarado’s critics. Nor would any PBS viewer even know that such critics exist. Instead, Smith crafts the kind of feel-good tale so typical of efforts like Making Schools Work—a tale in which all claims of “reform” are presumed to be right, in which some schools whose test scores are really quite low are said to be authoring “small revolutions.” The viewer feels good when the story is done. But have the interests of low-income schoolkids been served? And what has become of our standards?

Of course, one could overlook some of this if Smith’s basic judgments were solid. Yes, Resnick’s role should have been stated—but if she’s actually right in her judgments, one might overlook this detail. So how about it? Did Alvarado really show “that you could make many schools bloom, not just a few?” And, to borrow the language from the Smith web site, did Alvarado hatch a “small revolution” in District 2—a small revolution with “enormous implications for public schools nationwide?”

The answer to that isn’t obvious. For the sake of semi-clarity, here is Smith’s core claim about the success of Alvarado’s tenure:
“MAKING SCHOOLS WORK” WEB SITE: By 1998, Alvarado’s last year as District 2 superintendent, 73 percent of the students are reading at or above grade level, up from 56 percent a decade earlier; in math, 82 percent are at or above grade level, up from 66 percent in 1988. Alvarado leaves District 2 for position with the San Diego school district.
Source: HPLC Technical Report (2000)
This claim appears as part of the web site’s “Chronology of Events” for District 2. With the rigor characteristic of this program’s work, the site doesn’t say what “HPLC” means, nor does it link to this vaguely-identified report. But we do see the program’s basic claim for Alvarado’s eleven-year tenure. According to Making Schools Work, 56 percent of District 2 kids were reading at or above grade level when Alvarado arrived. Eleven years later, the rate was 73 percent. A comparable gain is cited for math.

Are these basic numbers accurate? We’ll assume they are, but we haven’t checked them; the site doesn’t say what test it is citing, and test scores for the period in question (1987-98) don’t seem to be available on New York City/State web sites. As always, some contradictory indications can be found. On April 26, 1989, for example, the New York Times reported, in a capsule profile of District 2, that “last year, 68.2 percent of pupils read at or above grade level” in the district—presumably meaning in 1987-88, Alvarado’s first year in the district. The figure seems strangely high, but who knows? The Times may have gotten the figure wrong, or Smith’s site may be citing some other measure. As we’ve seen, Making Schools Work is often quite expert at wringing good news out of any test figures—at finding ways to make schools (seem to) work, even where indications are gloomy. But we’ll assume what we (on balance) believe—that these are actually accurate numbers gleaned from some reading test or another. We’ll assume that, on some reading measure, District 2's rate in reading went from 56 to 73 percent during Alvarado’s tenure. That would seem like a hefty gain. But what might those numbers really mean?

In the first place, we’ve already seen that numbers like this can sometimes mean next to nothing. When Smith’s team whistled their way through the state of Washington, for example, they told us that Centennial Elementary had seen “a remarkable jump” in its math passing rate. “Only 23 percent of children met the state math standards in 1997,” Smith explained. “By 2004, that number more than doubled to 56.6 percent.” But when we took a closer look, we found that scores had jumped on the test in question all across the state of Washington—that the statewide gains had actually been larger than Centennial’s, suggesting that the state was simply administering an easier test at this point. The key point: Gains in test scores only have meaning if we know that the test in question has maintained its level of difficulty over time. We don’t know that about the New York test in question, whose identity has been left off Smith’s site.

On the other hand, District 2 does seem to have raised its scores more than other districts. Where was District 2 when Alvarado arrived? “Compared to the other 32 districts in New York City, District 2 ranked 10th in reading and 4th in math” in 1987, Smith’s chronology says. During the actual show, Smith puts his thumb on the scale a bit, then reports where the district ended up:
SMITH: What really got people’s attention was Alvarado’s success in all kinds of schools—higher scores in reading and math across the board. He lifted his district from average performance to the second best district in New York.
Question: If you’re tenth and fourth (out of 32) in reading and math, does that make you an “average” district? As always, Smith puts his thumb on the scale as he describes poor barefoot-and-pregnant District 2 back in 1987. But his basic claim is clear, made at various places on his site: During Alvarado’s tenure, District 2 went from tenth in reading and fourth in math to number 2 (out of 32) in both subjects. This would mean that District 2's scores were rising faster than others.

Does this mean that Alvarado achieved great success? Careful! If we care about truth (as opposed to pleasure), we’ll try not to move quite so fast.

First, let’s note a couple of things about Alvarado’s district. For the record, District 2 is not the typical New York City district. In this paper from the Education Policy Analysis Archives, Dr. Lois Weiner questions claims about District 2's alleged successes—and she notes, in scholarly language, that District 2 ain’t your average district. (“Another key fact about [District 2] that is not fully addressed in reports by researchers who promote it as a model is the district’s access to human and material resources that urban districts typically lack.”) In her piece, Weiner questions claims about District 2, including claims by Fink and Resnick; she specifically warns that researchers are becoming “cheerleaders” for District 2's reforms. (Weiner’s title: “Research or ‘Cheerleading’? Scholarship on Community School District 2, New York City.”) District 2 has lower poverty indicators than New York City as a whole, she notes, and its student demographic is substantially different from the typical district. “[District 2] differs demographically from other New York City districts, especially those with low levels of student achievement,” she writes. “The largest minority population in [District 2] schools is Asian, and the Asian and white population combined constitute 65 percent of the students served. In New York City schools, the combined Asian and white figure is 27 percent.” This means that District 2's African-American and Hispanic population is far below that of the typical New York City district. Since black and Hispanic kids tend to score lower than whites and most Asians, this suggests why the district was above average even when Alvarado arrived.

These facts provide some basic background. But they wouldn’t explain the central point—the subsequent rise in District 2's relative standing under Alvarado. If the district went from tenth to second in reading, wouldn’t that mean that Alvarado’s reforms really did produce success? Once again, it isn’t that simple, for reasons Weiner’s piece suggestst.

Let’s leave the dry-as-dust world of academia and return to Andrew Wolf’s pungent columns. Wolf is an open critic of Alvarado’s pedagogy (for ourselves, we have no view). But in his recent New York Sun column about Making Schools Work, Wolf rolled his eyes at one set of score gains:
WOLF (10/7/05): Mr. Smith's program is hard to pin down. It is rare that two hours of prime time, even on public television, is devoted to education, so much more the shame of this missed opportunity. But it appears that Mr. Smith has bought into Anthony Alvarado and the mythology surrounding the "District Two Miracle," hook, line and sinker. The school Mr. Smith highlights as making incredible gains is in Chinatown. There is no miracle at play in coaxing high performance from a school with an increasing population of Asian students.
Uh-oh! This is facile (as are most claims in 800-word columns), but Wolf suggests that District 2's “miracle” is related to changes in the district’s population—changes which Weiner also discusses in her scholarly piece. Did the district’s scores rise under Alvarado because of changes in the student population? We don’t know—Weiner told us this week that the relevant data aren’t available—but there’s no doubt that demographic changes occurred, and if we want to trade the pleasure of fable for the hard edge of truth, we’ll surely want to examine this question. In fairness, Smith’s team may feel they have answered this question—but their program doesn’t address it. Instead, they tell us that a district which scored fourth in math (out of 32) was only “average” when Alvarado arrived, heightening the viewer’s pleasure when the district jumps to number 2 as he leaves.

What kinds of changes occurred in the district? We can’t provide a full account, but some changes occurred because of Alvarado’s active recruitment of new students. Various people judge this policy various ways, but it clearly brought new students into District 2—many of them middle- and upper-class kids who were likely to be high scorers. In March 1992, for example, the New York Times’ Lynda Richardson described the way the district was attracting new kids—and improving its test scores:
RICHARDSON (3/20/92): District 2, which was ranked 11th in reading scores in the city five years ago, now ranks fourth out of 32, tying with Staten Island.

And enrollment has leapt up by about 1,000 schoolchildren, of which 55 percent are white middle-class, according to District 2 officials. They see the trend as stabilizing the flight of better-educated parents from the district.

Patti Seidman, an architect who lives in SoHo, said she might have enrolled her two young children in private schools instead of District 2 if not for Mr. Alvarado's programs. "He's the best thing that ever happened to District 2," she said.
Alvarado was bringing in kids who would otherwise have attended private schools. This may be judged as an admirable practice (critics point to certain problems), but resulting score gains would not show that District 2 was finding ways to raise achievement among low-income black and Hispanic kids. No, Alvarado couldn’t get from 56 percent to 73 by bringing in a thousand new kids (he started with about 17,000). But as the 1990s continued (the Times called it the age of “the school choice wars”), Alvarado—to his credit or detriment—seemed to keep finding ways to bring new kids to his district. In September 1992, the Times published an article about parents who were faking addresses to get their kids into desirable schools and districts—and once again, the paper noted the heavy influx into District 2. To what extent did this tilt the scores of the district? We don’t have the slightest idea, and Weiner says the relevant data aren’t available. But to the extent that you’re able to bring in new, high-scoring kids, it isn’t hard to raise district test scores. This doesn’t speak to the critical question: Have you found ways to raise the scores of your pre-existing low-scoring kids?

There are suggestions that Alvarado may not have done that. As we’ve already seen, Daria Rigney described P.S. 126 as a pitiful wreck when she arrived in 1998, after Alvarado’s eleven-year tenure. (To keep this fact from clouding his story, Smith pretended that Rigney was describing the school as Alvarado himself had found it.) And as we’ll see in our next post, School 126 still had very low scores at the time Rigney took over. It was full of poor kids who were still doing poorly—after Alvarado’s reforms.

And uh-oh! A second report from the New York Times suggests another low-income fine mess. In 1996, Somini Sengupta described another low-income school that was floundering under Alvarado, nine years into his run:
SENGUPTA (12/8/96): For years, Community School District 2 has been in the limelight: Its superintendent, Anthony J. Alvarado, has been lauded for setting up small, experimental schools with innovative approaches to teaching.

Now, for the first time, a District 2 school has come under a harsher spotlight. The school, Public School 198 on East 96th Street between Lexington and Third Avenues, recently landed on the State Education Department's list of the city's worst-performing schools because only 26.2 percent of its third graders read at or above grade level.

District officials say there is no one simple reason for the poor performance. But Andrew Lachman, a spokesman for Mr. Alvarado, noted that P.S. 198's teaching staff is relatively inexperienced, which he said accounted for some of the underachievement. More than half the school's 25 teachers have been teaching for less than five years, Mr. Lachman said, compared with 25 percent citywide, and more than a third have spent fewer than two years at P.S. 198. "They may not have the repertoire yet," he said.

There are other factors that could influence performance, though P.S. 198 is better off than some schools in these areas. While 86 percent of P.S. 198's pupils are poor enough to qualify for a free lunch, all students at several other schools—in East Harlem, Chinatown and Chelsea—are eligible. Nearly one out of five P.S. 198 pupils have only limited English skills, but the portion in some other District 2 schools is greater.
Politely, Sengupta avoided the point which Wolf suggested; low-income kids of Chinese ancestry tend to score a good deal better than low-income black and Hispanic kids. But here, as at P.S. 126, Alvarado’s reforms didn’t seem to be doing much for those low-income black and Hispanic kids—the kinds of deserving, low-income kids our public schools must find way to serve better. Was this typical of District 2 as a whole? We don’t know, and Weiner says the relevant racial data aren’t available from this period. But schools 126 and 198 suggest that the possibility that Alvarado’s reforms were attracting high-scoring new kids into the district—but failing to provide much help for the district’s low-income black and brown kids, whose overall numbers were fairly small by New York City standards.

In Making Schools Work, Hedrick Smith went looking for schools and districts which were achieving “small revolutions” with deserving low-income kids. Did District 2 do this under Alvarado? We don’t have the slightest idea. But then, when you watch this PBS show, you don’t even know that Lauren Resnick is praising her own work at the start of the District 2 segment. Do we care about the truth in these matters? Or are urban kids just entertainment for those “cheerleaders” Weiner derides?

FOR THE RECORD: Resnick replied to Weiner. In part, she said this:
RESNICK: One HPLC analysis examined changes over time in reading and mathematics during the period 1992 to 1998—a period in which the CSD2 curriculum and professional program was being put into place and expanded, and during which a stable test in each subject was being used in New York City. In 1993—the first year of New York’s renormed math achievement test, just under 70% of CSD2 students were at or above grade level in math; in 1998 , about 82% of CSD2 students were at or above grade level. The story is similar for reading: Scores rose from just under 60% at or above grade level in 1992 to about 72% in 1998.

This gradual rise in overall achievement could have resulted from a change in overall district demographics resulting from more middle class students attending CSD schools. But it did not. According to our data, during this period the percentage of students in the district eligible for free or reduced lunch remained stable at about 53%.