Charter schools may be the best thing for education, but there is no way to know that.

We are in the middle of a grand experiment in public education. Charter schools are opening everywhere, including ones specializing in the arts, sciences, and technology. Outsource programs such as Renaissance Schools and Knowledge Is Power are taking over large segments of inner-city systems. Private, not-for-profit organizations have become the "savior" of our "failing school systems."

But as with any experiment, we need to be able to evaluate the results, which so far appear mixed. Pennsylvania's revised method of evaluating charter schools reported that only 28 percent of the more traditional charter schools and none of the state's Cyber Schools (another outsource) met the standards for "adequate yearly progress."

Are charter schools the pathway to the future or fool's gold? Are our public schools failing, and if so, which ones and why? Should we use a combination of approaches? Unfortunately, we have absolutely no idea of the answers because we don't have an accurate method of determining success or failure.

Measuring and comparing school performance is extraordinarily difficult and the results might not be intuitive. A charter school scoring higher in standardized tests - higher than the traditional public school that sends students to the charter school - might not be succeeding. Conversely, a Renaissance School with low test scores might be doing extremely well. It all depends upon how you determine performance.

Consider 'input'

How is it possible that a school that has high test scores is failing while one that has low test scores is succeeding? As an economist, I call that "looking at the output without considering the input."

The quality of a good that is produced depends upon the production process, which includes the type of inputs and efficiency of the factory. If you start with high-quality materials and put them through a high-technology manufacturing plant, you should get an outstanding product. Anything less probably means you are not doing a good job.

If you start with low-quality inputs in an old plant, a product of mediocre quality might mean you have done a great job with what you were given.

The idea that it is the educational inputs (students and parental involvement) coupled with the factory (the school facilities and management and the quality of the teachers) that produces the educated child means that we have to consider all the input and production factors when comparing schools.

It is not enough simply to evaluate test scores - the output.

And that is where the complexity of measuring student performance comes in. Multiple factors determine how well a student does. Intelligence is just one. Income, especially as it relates to things such as living and health conditions, is important. Tired, hungry children have trouble succeeding regardless of their basic intelligence or drive. Supportive parents who are involved with schools also matter.

Environment counts

But it doesn't stop there. The school itself, the factory, is critical. Is it safe, does it reduce stress, and do the students feel involved? Does it provide for the needs of special-education students and those with special talents? Does it have adequate technology and supplies? And last but hardly least, are the teachers involved and effective? A large set of factors goes into determining the success of a child in school.

To evaluate the relative performance of public vs. alternative school models, you have to adjust the test scores for the differences in all those factors that determine success. That is especially true because of what researchers call "selection bias." Parents have to choose to send their children to a charter school, meaning that charters start with students who have high parental involvement. If the charter school is newer or safer or has more technology, the difference in performance is expected to be significant.

Unfortunately, we don't adjust test scores for differences in factors such as intelligence, income, parental involvement, or school facilities. Instead, we use misleading, unadjusted test-score comparisons. That means some schools that test well might not be doing all they can, while those with failing scores might be making outstanding progress. We are largely clueless about what is happening.

Charter, Renaissance, Cyber, and other alternative school models are great ideas and may turn out to be the greatest thing for education since sliced bread. Or, they could wind up being a total disaster.

Unfortunately, we have no way of determining whether this massive educational experiment is succeeding. That is unfair to the new school models, the public schools, the taxpayers who fund all the educational approaches, and most important, our children.