Five middle school students in Elizabeth, Pa., stand in a darkened room over a pie chart projected on the floor. Using motion-sensor wands, they manipulate sections of the circle, making pieces representing one-eighth and one-twenty-fourth.
In the Ridley School District, students sit in the hallways tapping out homework assignments on their cellphones to submit through cloud-based software.
Such technology-infused classrooms are popping up around the country. Proliferating with them are new gadgets, new apps, new teaching programs, and new devices - some expensive, all relatively untested.
As school leaders buy new technology, they're seeking to do more than just replace paper with laptops. They want products designed to be part of a philosophical shift in education, one that enables students to direct their own learning, and teachers to coach and encourage them. Technology is seen not only as an outlet for creativity, but also as a way to engage kids raised in a culture of YouTube and Minecraft.
A growing number of companies claim their products can fill this need. Yet school leaders face a daunting challenge: From the slew of highly touted products, how do they pick the right ones?
"It's hard for our people to know what all of the choices are," said Penny Hodge, assistant superintendent of budget and finance in Roanoke, Va. Today's school leaders must navigate a market with little trustworthy evidence to show what works. Billions of dollars are being spent while educators try to untangle a maze of sales pitches.
"I'm faced with that on a regular basis now," Ridley Superintendent Lee Ann Wentzel said of the promises by ed-tech providers. Her district, in Delaware County, is an Apple Distinguished Program because of its innovative curriculum, using iPads in all the district's classrooms. "People know we have technology, so they say, 'This is the solution that's going to solve all your problems.' "
Even in the Philadelphia schools, where a cash-strapped budget has leaders focusing on free technology solutions, like Google Apps for Education, there must be evidence and thorough vetting before moving forward. Leaders offer a curated list of options for teachers and guidelines on how teachers should choose apps to use for learning.
"If it was free and not high-quality, we would never bring it to teachers," said Fran Newberg, Philadelphia's deputy chief of educational technology.
Picking effective products is a national challenge. A survey of 300 U.S. school leaders and technology executives by Digital Promise, a congressionally authorized nonprofit that helps schools adapt to new learning technologies, found two persistent themes: Schools need help finding the best technologies; and when good products are found, schools need a better way to share that information.
If more efficient pathways don't emerge, schools may spend millions of dollars and many valuable hours of staff time on the wrong technologies, while the best innovations may go undiscovered and opportunities to transform classrooms may be lost.
"Everybody's on the losing end of the current system," said Eileen Murphy Buckley, founder and CEO of ThinkCERCA, a company providing personalized learning technology to schools. "The innovation has been stifled . . . and school districts then don't have the selection and competition that would help them get the best products."
Last year, American schools bought $8.38 billion in software, digital content, or training and assessments, an annual increase of more than 5 percent, according to Karen Billings, vice president of the Software and Information Industry Association's Education Technology Industry Network.
The digital education movement aims to infuse technology into everyday teacher practices and encourage student-driven learning.
Yet as school leaders struggle to create these new kinds of classrooms, many are not using empirical evidence to make technology choices. They're using word of mouth, school visits, and the results of "questionable" pilots, according to the November report "Improving Ed-Tech Purchasing," by Digital Promise and the Education Industry Association.
Part of the reason is that credible evidence often isn't available. Only one-third of school technology directors surveyed said that education technology companies offer reliable data on their products, according to the survey.
Some educators get sold on a pitch alone.
"Educators don't make strong, evidence-based decisions as much as they should," said Thomas Ralston, superintendent at Avonworth School District, a small district near Pittsburgh. "I have teachers in the district who love a program, but when you tell them the data's not showing results, they don't want to see it."
In Philadelphia, leaders are studying how to give teachers and principals guidelines on choosing and evaluating the programs they use.
"We've been delving deeply into this; there's not a clear-cut, easy answer," Newberg said, noting that educators need to consider not only efficacy but also student privacy concerns. "We are talking to other districts. We are all asking the same questions."
The Philadelphia schools are examining technology to support struggling students and options to allow teachers to do blended learning in their classrooms. Both searches involve teams of educators evaluating requests for proposals, interviewing vendors and validating data claims.
"It's a very difficult process," Newberg said.
School district leaders and private companies alike say they want to see a reputable curation website, with professional reviews and a social-media component. They also need models for how to pilot products, since many current trials are not structured well enough to be trusted by other districts, according to the Digital Promise report.
Such things are in the works.
A website that collects third-party app reviews, called Graphite, was launched by Common Sense Media in 2013. There, teachers can find education experts' reviews of thousands of apps. The site also lets teachers add "field notes," telling others how they used a particular app in their classrooms.
In addition, the Bill and Melinda Gates Foundation has funded fast-paced technology trials, attempting to pave a new way to get reliable feedback from the classroom. The quick-evaluation pilots in New York City, Chicago, and the San Francisco Bay Area are giving teachers products handpicked by experts. Researchers then document, analyze, and report on results in the classroom.
The Washington-based Digital Promise also is researching best practices for pilots in schools. It will soon launch an initiative to study schools as they develop blended learning programs and then publicize the results, said spokesman Jason Tomassini.
Those convinced that technology will improve learning believe that taking risks is the only way to move schools forward. They say that the growing pains of the market are just that - kinks in an evolving system that will find its footing as the movement for education technology expands.
Bart Rocco, superintendent of the Elizabeth Forward School District, believes that schools should become incubators for technology. Several of the district's successful initiatives came into being because developers and teachers worked collaboratively on solutions.
"We've got to change the model and break down those walls," he said.