AI and the illusions of college teaching

How will artificial intelligence affect student learning on college campuses?

We don’t know. And here’s the worst part: We don’t want to know.

That’s because we have never taken teaching — or learning — seriously. We don’t prepare future professors for the classroom. And have no real systems for evaluating and rewarding their instruction.

So we also don’t have good ways to evaluate the gee-whiz rhetoric from the AI industry, which assures us the tool will “revolutionize” learning. We’ve heard that song before. In 1913, Thomas Edison forecast that books would “soon be obsolete” in schools, and that “every branch of human knowledge” would be taught via film.

Like today’s AI barons, Edison had a vested interest in that outcome: He helped invent motion pictures and started a company to market them in schools. But none of his predictions came true. Ditto for the futurists who said radio, television, and personal computers would transform teaching and learning forever.

But AI is different, its enthusiasts tell us, because it interacts with us. In the brave new world to come, everyone will be taught by a robot tailored to their individual interests, skills, and goals. It will help students “build agency,” “learn continuously,” and “solve hard problems,” declares the largest provider, OpenAI.

The initial research suggests the opposite: Students forsake their agency to the bot when challenges arise. As a recent report by an AI committee at the Massachusetts Institute of Technology (MIT) warned, AI creates “the illusion of learning” for students who use it “at the first hint of struggle.”

But in higher education, we’ve been fostering the illusion of learning for a very long time. Over the past half century, we have assigned less work and awarded higher grades. And there’s no evidence — none — that our students are learning more.

Nor have we made a sustained effort to incentivize teaching. At every type of institution, from small community colleges to huge private universities, professors who spend more of their time on research make more money. Those who devote themselves to teaching make less.

And nobody is really minding the classroom itself. I have been teaching at the University of Pennsylvania for 10 years, and I have never been observed by someone with supervisory authority. I could be doing anything. Or nothing.

We want AI to “enhance student learning,” not inhibit it. But we never bothered to figure out how much students were learning in the first place.

Sure, my students submit evaluations of my courses. But the results have never figured into my rank or salary. And a good review from your students doesn’t mean you did a good job teaching them.

Indeed, it might mean the inverse. In a 2010 study at the Air Force Academy, where everyone takes the same courses, professors who got high marks from their students gave out higher grades — and their students did worse in subsequent classes. Professors who were harder graders got lower student evaluations, but their students performed better later on.

If we really valued teaching, we would evaluate it the same way we judge each other’s research: via peer review. When I wrote a history of college teaching, I didn’t submit it to a group of 19-year-olds to decide if I had something important to say. It was vetted by specialists in my field.

We would also give future professors sustained instruction in how to teach. To obtain a doctorate at Penn, you have to spend six to nine years reading in your field and conducting research that contributes to it. To become a teaching assistant, by contrast, you receive a three-day training.

That’s why I called my book The Amateur Hour — not because all college teaching is terrible (some of it is terrific), but because we never professionalized it. We don’t hold each other to best practices, and we don’t have good data about who is doing what.

That’s the biggest scandal in higher education we talk the least about. And AI has shone a bright light on it.

Dozens of institutions — including Penn — have released policies and statements about proper and improper uses of AI. We want AI to “enhance student learning,” not inhibit it. But we never bothered to figure out how much students were learning in the first place. So we’re in no position to judge how the bot is affecting them.

Every institution providing Claude or ChatGPT to its students should also commit to researching their learning. It won’t be easy or cheap. But it’s the only way to see our way out of this mess. Without real knowledge, we’ll be flying blind.

I’ve heard colleagues say teaching and learning are too subtle and complicated to evaluate in a meaningful way. Please. Scholars at Penn have examined hugely complex forms of human behavior, from consumer choices to sexual decision-making. If we wanted to do the same for what happens in our classrooms, we could.

Will we?

Jonathan Zimmerman teaches history and education at the University of Pennsylvania. He is the author of “The Amateur Hour: A History of College Teaching” (Johns Hopkins University Press).