One of Wharton School professor Cade Massey's recent research projects started as a consulting deal with a National Football League team.
The franchise asked Massey, "Who is best at the draft? Who should we be paying attention to?"
His answer was surprising.
"I went out and looked at the data," said Massey, sitting in his office on the fifth floor of Huntsman Hall at Penn.
"It turned out, there are no differences in teams' abilities to draft."
There are clearly "huge differences" in outcomes, Massey was quick to point out.
"Some teams have great years, other teams have bad years - and it matters," Massey said. "But those differences aren't persistent year-to-year, which tells me that they are chance driven. Something between 95 and 100 percent - I'm not exaggerating - of team differences in the draft is driven by chance."
There is skill involved in selecting players, the professor said.
"It's just that teams are equally skilled, in a very uncertain environment," Massey said.
That eye-opening conclusion probably won't be received with a lot of "darn rights" within the NFL. But Massey, who previously worked at Duke and Yale, and coauthors NFL power rankings that were published in the Wall Street Journal, has punctured NFL balloons before.
An earlier paper he cowrote, titled "Loser's Curse: Overconfidence vs. Market Efficiency in the NFL Draft," basically flipped established beliefs about the worth of NFL draft choices. It got plenty of attention across sports.
On a basic level, this all makes sense. How much credit do the Patriots deserve for drafting Tom Brady in the sixth round? If they'd known Brady was going to be half as good as he turned out to be, they obviously wouldn't have let everyone else have a crack at him for five-plus rounds.
But Massey's work goes deeper than that, into the real value of picks. For years, NFL teams based the worth of draft choices on something that came to be known as The Chart, and gave a value to each pick.
Except, according to research data collected by Massey and coauthor Richard Thaler, it was all wrong. Top choices were overvalued once you factored in their salaries.
"The genesis of that paper is the '99 draft," Massey said. The consensus was that Kentucky quarterback Tim Couch was the top pick, as he turned out to be. Of course, Couch didn't turn out to be the top QB. That was the year Donovan McNabb was drafted.
"We thought when you looked at '83, the lesson should have been, you don't know which of these quarterbacks are going to be the good ones," Massey said. "[Dan] Marino was the sixth [QB] taken. [John] Elway was first, but there's [Tony] Eason and [Todd] Blackledge in between."
For the football-history impaired, Marino was a bit better than Blackledge or Eason. But Massey's point isn't about scouting ability. (Alleged character issues were the reason for Marino's fall in the draft.) It's about overconfidence. "They're too sure that they can predict the future," he said.
That's Massey's real interest. The NFL draft happens to provide the data.
"This isn't about the NFL draft," he said.
His field, Massey explained, has shown very clearly that people don't appreciate how much uncertainty there is in life. And now he's trying to come up with strategies that "are optimal when the environment is more uncertain."
Massey worked as a long-term consultant for an NFL team that didn't want to be identified, he said. (Not the Eagles.) He is friends with plenty of analytic types, including new 76ers general manager Sam Hinkie, whose brains and creativity he raves about.
Massey is offering ideas that aren't wedded to sports alone.
"You should play lottery environments very differently than math-problem environments," Massey said. "With math problems, there's a right answer. Work hard enough and you'll get the right answer. If you can't get it, find the smartest guy in the room and he'll have the right answer."
"You don't spend any time trying to get the right number because it doesn't make any difference," he said. "You try to get as many draws as possible or spend as little money as possible."
One of Massey's central tenets: People tend to think of life as more math problem than lottery. There's some mix, obviously. But they've got the ratio wrong. Given the strong element of chance, he said, you don't throw up your hands and give up.
The prescriptions are very pedestrian, he added.
"As many draws as possible. Minimize cost. Value the process," he said.
That last one is interesting. If there is more uncertainty, Massey believes, the process should actually be valued more.
"You talk to some teams - these guys put in months and months, the scouts do, building the profiles of all the players," Massey said. "They build the draft boards. And then some guys, some owners, some GMs, will come in late in the process, even draft day, and shuffle some of these boards, just completely violating the process."
That would be fine, Massey said, if there were a right answer.
"But in a situation where there is no right answer, you're completely flouting a process, which is going to screw up the organization," Massey said.