So much for “The Jetsons.” Flying cars, floating cities and robot servants never happened. A decadeslong slowdown in “disruptive” discoveries, detailed in a new report looking at scientific studies and issued patents, might be one explanation.
For more than a decade, scientists and economists have worried that groundbreaking inventions have become fewer and farther between. Where the early 20th century delivered novel innovations that ranged from airplanes to atomic bombs, more recent years have instead seen an “innovation slowdown” in the rate of invention, even in the face of ever-growing investments in research.
“A lot of the funders behind science and technology have been arguing or proposing that we need more disruptive science and technology to deal with climate change and other societal grand challenges [as] just kind of a way to address these very important problems,” said University of Minnesota innovation researcher Michael Park, an author of the new analysis in the journal Nature. Even though the raw number of papers and patents have exploded in the last half-century, said Park, “the percentage of disruptive work is declining across all major fields.”
The slowdown — seen across 45 million papers and 3.9 million U.S. patents stretching from 1945 to 2010 — may point to even bigger problems in the economy and cultural scene, say some observers. About half the economic gains in productivity come from innovations in science and technology, for example, but productivity growth has slowed in advanced economies. In the last 15 years, it has increased at just half the rate it did in the 15 years before then. Striking declines in “disruptive” works, ones that “break with the past in ways that push science and technology in new directions” in fields from physics to medicine, might just be a symptom of this bigger slowdown.
“You’re not crazy — this is actually happening,” said Johan Chu, an assistant professor of system dynamics at the MIT Sloan School of Management, who was not part of the study. “We are living in a world which is very different from the 1950s, where you could make fundamental advances, to a world where that has become very, very hard to do.”
Overall, it is taking more and more papers and patents to produce a truly disruptive one, according to the analysis. The finding rests on citations to earlier work found in both scientific papers and in patents, to give them a disruption “index.” In the case of groundbreaking ones, later works that cite them are less likely to cite the works the groundbreaking papers reference because they made it obsolete. (An example is a widely promoted model of DNA as a “triple helix,” now forgotten, that preceded the 1953 discovery that it instead possessed a double helix structure.) Non-disruptive work, in contrast, will see plenty of later citing of its references, because it hasn’t made them irrelevant.
In scientific disciplines, declines in disruption indexes from 1945 to 2010 ranged from a 91.9 percent drop in social science papers to a 100 percent dive in physics studies, with a steep plunge until the 1970s and then a slower decline that levels off by 2010. In patents, from 1980 to 2010, the decrease ranged from a 78.7 percent fall in “computers and communications” to a 91.5 percent drop in “drugs and medical” categories, both following a similar curve. “Our big contribution is confirming these sort of suspicions that people had and showing it across all major fields of science and technology,” said Park (who acknowledged, with some irony, that this means his team’s disruption finding is thus not “disruptive” by its own measure, but rather is a “consolidative” report).
Backing up the results, the study found mirroring declines in the language used in papers and patents over time, with fewer introducing new words, and verbs like “make” and “produce” appearing less frequently, while ones like “improve” and “enhance” turn up more often. More and more scientists and inventors are doing more and more work along narrower and narrower avenues of knowledge, conclude Park and his colleagues, and producing groundbreaking work much less often.
“This paper is a monumental achievement,” said sociologist Bas Hofstra of Radboud University in the Netherlands. What is particularly convincing, he said, is that the evidence stretches across a varied archives of studies over time, which means it accounts for the effect being driven by arbitrary citation practices and appears statistically robust. “I am sure this paper will lead to many spinoff studies,” Hofstra added, trying to find scientists and inventors who don’t have declining disruption indexes.
In response to concerns about fewer groundbreaking discoveries, industry and government have turned to prizes, such as the XPrize competitions, to spur innovations. The U.S. National Academies of Sciences, Engineering and Medicine has responded to the slowdown by proposing “Grand Challenges” for technologists to overcome. And the federal government has turned to initiatives like the “Cancer Moonshot” at the National Cancer Institute, or erecting innovation offices within agencies meant to emulate the Defense Department’s Defense Advanced Research Projects Agency (DARPA), which funds high-risk research, and seeded stealth, drone and early internet technology.
But just why there has been an overall decline in breakthroughs as a share of all research and patents isn’t exactly clear.
One popular theory for the decline in productivity and innovation seen in recent decades is that all the “low-hanging fruit” has already been taken in the boom years for science after World War II. To put it another way: A field can make its fundamental discovery only once. That would make it understandable that researchers now need to pedal faster just to keep making groundbreaking discoveries.
The medicinal chemist Derek Lowe favors this view, writing, “The effect is still going to apply to any field that’s actually making measurable progress,” in a critique of the disruption index paper on a blog published by Science. “I’m not willing to ditch the low-hanging-fruit explanation as part of the story.”
But the study argues that all the disciplines studied in the paper have seen similar declines in disruptive reports falling at roughly the same rate over the decades in the same way. Low-hanging fruit is unlikely to have all been picked at the same time in the same way in all those disciplines across the decades. Instead, Park and colleagues suggest that researchers and inventors are drowning in the explosion of knowledge that followed World War II, which has come too fast and furious to keep up, forcing them into narrower and narrower lanes of investigation that can be mastered to look for discoveries. From 2008 to 2018, for example, annual research study output in one global study database grew from 1.8 million to 2.6 million studies, a 4 percent yearly increase rate.
Patent practices today might explain the disruptive index findings in that area, said Vincent Paolo Violago of Parola Analytics, Inc., a New York-based patent research firm. Instead of patenting one fundamental innovation, inventors now repeatedly patent improvements to retain licensing rights, he said: “I really wouldn’t expect much disruptive technology from patents because patents have a more economical use than that of reporting new discoveries.”
Violago called the study results more disturbing, suggesting that funding might be the issue: “No one really does research these days out of their own pockets,” he said. Funding agencies and foundations are going to favor more conservative, assured results, rather than moonshots, and journals are more likely to greenlight something that looks familiar. Scientists often complain that winning a lucrative grant from the National Institutes of Health, the big prize in biomedical research, requires that half the research is already completed, leaving little time for blue-sky experiments. And it drives grants to already-funded labs that will look to continue older work rather than for new scientific veins to mine. “With this type of culture, it’s actually easier to publish a paper on something that is built upon an earlier research instead of working on something really new,” Violago said.
“If you wanted to design a system that prevents innovation, you’d probably come up with something similar to the way science works right now,” said experimental psychologist Adam Mastroianni of Columbia Business School, in reaction to the study. “Make scientists spend half their time asking for money. Don’t publish anything until it has satisfied reviewers’ whims. Exclude hobbyists, amateurs, and anyone with strange ideas. And most importantly, make the whole thing so competitive that everybody’s terrified of taking any risks. That system will produce lots of papers, but little progress.”
The study authors call for funding agencies to provide more sabbaticals and funding of long-term, undirected research to scientists to combat funneling them into low-risk, low-payoff experiments. “There’s more work that needs to be done to really understand different factors that are contributing to this trend,” said Park.
But such fixes look more like Band-Aids than solutions, said MIT’s Chu. He sees the trend described in the paper as a symptom of a bigger malaise, reflected not just in slowed economic productivity growth, but also in the superhero movie sequels that fill theaters, rather than original works, and the video game sequels that likewise top sales expectations. “We live in a world where we have an abundance of choices of how we spend our time,” said Chu. Like the scientists overwhelmed by the burden of knowledge in their discipline driving them to smaller and smaller specialties, everyone faces a similar burden in business and everyday life.
“I think there was a period perhaps around the Second World War or a little past then when devoting more resources toward creating more things was the right answer and it worked tremendously well in terms of science and in terms of everything else,” said Chu. “But you get to now, and we’ve kept that sort of same industrial mindset of ‘more is better,’ and we have more scientists and more papers and more journals and all these things. But at the end of the day, having all that much more doesn’t seem to add that much more disruptive or far-reaching knowledge.”
It’s not just endless Marvel movies that are the result, but the expanding American workweek, where two-earner families work longer hours, with the average workday exceeding the eight-hour one set as a standard in 1938 by the New Deal, according to the Bureau of Labor Statistics.
Private funding of research that allows for longer-term research agendas might help scientists make more groundbreaking discoveries, said Chu, “but the jury is still out.” Corporate research labs such as Bell Labs, which pioneered transistors, cellphones and internet technology, have largely vanished in a corporate world driven by quarterly stock prices, drying up such oases of long-shot research. There is also some hope that artificial intelligence-assisted literature searches will tip researchers to potentially groundbreaking results or patents obscured by the flood of marginally more important knowledge.
The silver lining to the disruptive index finding, said Park, is the study finding that groundbreaking studies and patents did keep appearing, with their slowdown flattening off after the 1990s to a steady level. That implies there is a baseline level of groundbreaking discoveries that will keep coming, whatever inefficiencies are built into the current research and patent enterprise. They keep popping up across disciplines, from mRNA vaccines during the pandemic, to the discovery of “dark energy,” an accelerating expansion of galaxies throughout the universe, in the late 1990s.
“It’s kind of a sign that we haven’t reached the end of the endless frontier,” said Park. “The frontier is still out there, and we can still keep pushing. If we are incentivizing scientists and investors in the right way, perhaps we could continue to push the borders of science and technology.”
Thanks to Lillian Barkley for copy editing this article.