Pennsylvania's top education official is concerned enough about the contents of a report meant to ferret out cheating on 2009 state exams, his spokesman said, that he will order further review and possibly action.
A Philadelphia School District official said that if directed, the district would reopen old cheating investigations and launch new ones.
A "data forensics technical report," prepared for the Department of Education in 2009 and first obtained by the Philadelphia Public School Notebook, analyzed schools statewide for possible testing improprieties.
Nearly half of the roughly 60 schools flagged for multiple statistical irregularities are in Philadelphia - 22 Philadelphia School District schools and seven charters.
Among the schools flagged is Roosevelt Middle School. As The Inquirer reported in May, multiple Roosevelt teachers said they witnessed many test security breaches and they attributed a remarkable two-year rise in state test scores to cheating.
The chances that the wrong-to-right erasure pattern of Roosevelt's seventh-grade reading exam booklets happened by chance were slightly less than 1 in 100 trillion, The Notebook reported.
"I can tell you the report concerns the secretary," said Timothy Eller, spokesman for Secretary of Education Ronald Tomalis.
The state will take a closer look, but it's not clear what form that will take, Eller said.
Tomalis was troubled in particular by districts and schools that were flagged for multiple irregularities, Eller said.
The secretary has also ordered that forensic analysis be reinstated, and it is being prepared for the 2011 PSSAs, which were administered in March. That report should be available this month, Eller said.
District officials said they had not seen the 2009 report, which examined tests administered that spring. Several charter operators named also said they were not told about it.
Spokeswoman Jamilah Fraser said the state has not notified the district of "any serious breach in our testing process," but said the district was prepared to reopen or start investigations as directed.
But Fraser said that was contingent on the district's getting answers to questions "including how to think about and use the results."
Fraser said that investigating tests from two years ago would be a challenge, but that if the state "has questions about testing irregularities in the 200-some-odd schools noted in the report, which includes Philadelphia, the district stands ready to assist in any way possible."
Eller said he did not know why the state did not act in 2009, but stressed that the report was prepared during Gov. Ed Rendell's administration.
Going forward, if schools are flagged for multiple statistical irregularities, the state will ask districts to investigate and forward reports to the Department of Education.
If the district's own investigation is deemed satisfactory, the matter will be closed, Eller said. If officials find that a deeper probe is warranted, the state will launch one.
Depending on the severity of the cheating infractions, the state's Office of General Counsel could act, Eller said.
"We want to ensure that the credibility and security of the testing is top-notch," Eller said.
Authenticating test results is particularly important now, Eller said, when officials want to tie teacher pay to student assessments.
The 2009 report flagged potential cheating in schools across the state and did not accuse anyone.
"Their scores, response pattern, and number of erasures were aberrant, from a statistical probability perspective," the report said. "This does not imply that the school or student engaged in inappropriate testing activity. The statistical evidence merely suggests that something aberrant or unusual occurred."
Roosevelt's PSSA scores rose 50 points in reading and 52 points in math between 2007-08 and 2009-10.
Teachers who spoke to The Inquirer say they saw numerous improprieties during that period, from test answers written on a blackboard to administrators encouraging teachers to drill concepts they knew appeared on the exam. Roosevelt teachers also said they saw administrators giving students back their books so they could correct wrong answers.
The 2009 state report says schools were identified for analysis "if they had a large change in scale score, percentage of proficient or advanced students, difference between their actual and predicted mean scale score, number of wrong-to-right erasures compared to the state mean, and subgroup participation rates."
Roosevelt was flagged by the state for both reading and math irregularities for both grades that attend the school.
A separate district investigation of the 2011 test - not a statistical analysis - concluded that allegations of cheating this year were unfounded.
Each year, the district investigates 10 to 15 complaints of possible cheating; 15 were received this year, a spokeswoman told The Inquirer.
Officials from four of the seven charters that received multiple flags - Wissahickon Charter, the Charter High School for Architecture and Design, Walter D. Palmer Leadership Learning Partners, and Imhotep Institute Charter - said they were unaware of the report.
The 11th-grade scores at Imhotep, in East Germantown, were flagged in seven of the eight possible areas.
M. Christine Wiggins, chief executive officer of Imhotep, said that the report without follow-up investigation does not point to any wrongdoing.
"When district test monitors have visited our school in the past, we have not received any negative feedback," Wiggins wrote in an e-mail. "We stand by the integrity of our staff who administer the PSSA (who receive annual PSSA administrator training) and are proud of the great strides our students have made as evidenced by our increased student achievement."
Her school saw a 28-point jump in 11th-grade math scores in 2009. Even with higher math scores, Imhotep did not meet state standards that year because of lower reading scores.
Fraser defended the integrity of the "overwhelming majority" of district staff. "Until we have reason to believe otherwise, we stand on the integrity of these individuals who educate our students throughout the year," she said in a statement.
State test scores have risen steadily over the last nine years in the district.
Schools in Report
These Philadelphia schools were flagged in a 2009 state Department of Education report for suspicious PSSA scores on three or more measures in one grade.
*Denotes schools with three or more flags
Elementary schools: Catharine, *Cayuga, Franklin Edmonds, Frederick Douglass, Disston, Fitler, Fitzpatrick, Robert Lamberton, *John Marshall, *Thurgood Marshall, McClure, *Munoz-Marin, Olney, M.Hall Stanton and Ziegler.
Middle schools: Penn Treaty, *Theodore Roosevelt, *Wagner and Woodrow Wilson.
High schools: Frankford, Northeast, Strawberry Mansion.
Charter schools: Alliance for Progress, Charter High School for Architecture and Design, Imhotep Institute, Maritime Academy, Walter D. Palmer Leadership Learning Partners, Wissahickon and Philadelphia Electrical & Technology
SOURCE: Data Forensics Technical Report, Pa. Department of Education & Philadelphia Public School Notebook analysis