Papers and patents are becoming less disruptive over time

Existing perspectives on disruptive science and technology

A disruptive idea is one that renders existing knowledge obsolete, propelling science and technology in new directions. While the 20th century witnessed an unprecedented expansion of scientific and technological knowledge, there are concerns that innovative activity is now slowing, with declining research productivity in semiconductors, pharmaceuticals and other fields. Papers, patents and even grant applications have become less novel and less likely to connect disparate areas of knowledge, both of which are precursors of innovation. The gap between the year of discovery and the awarding of a Nobel Prize has also increased, suggesting that today’s contributions do not measure up to the past. These trends have attracted increasing attention from policymakers, as they pose significant threats to economic growth, human health and wellbeing, and national security, along with global efforts to combat grand challenges like climate change.1   

Why the slowdown?

Numerous explanations have been proposed. Some point to a dearth of “low hanging fruit” as the readily available productivity-enhancing innovations have already been made. Others emphasize the increasing burden of knowledge; scientists and inventors require ever more training to reach the frontiers of their fields, leaving less time to push those frontiers forward. In addition, the tenure promotion process and the competition for research funding in academia may influence the type of research with which scientists are incentivized to engage, in particular favoring lower-risk incremental pursuits. Overall, much remains unknown, not merely about the causes of slowing innovative activity, but also the depth and breadth of the phenomenon.

The decline is difficult to reconcile with centuries of observation by philosophers of science that prior knowledge enables future discovery – a view captured famously in Newton’s observation that if he had seen further, it was by “standing on the shoulders of giants”. Moreover, the current evidence of the slowdown is based on studies of particular fields, using disparate and domain-specific metrics, making it difficult to know whether it is happening at similar rates or for similar reasons across different areas of science and technology. This also makes it difficult to tell whether the aggregate patterns mask differences in the degree to which individual works push the frontier.

Quantifying the decline in disruption

One way of assessing the disruptiveness of an idea is to look at patterns of citation. If a new idea simply consolidates existing knowledge, it is likely to be cited alongside its predecessors. If it is disruptive, however, it should displace its predecessors and make them irrelevant for future studies. Assessing papers based on citation patterns produces an index of consolidation versus disruption (CD index). Allowing time for new knowledge to be disseminated and assimilated, the CD₅ index measures disruptiveness by allowing five years to elapse post-publication.

Our sample consists of 25 million papers published from 1945–2010 in the Web of Science (WoS) and 3.9 million patents from 1976–2010 in the United States Patent and Trademark Office, which encompasses scientific and technological discoveries from the public and private sectors such academic institutions as well as for-profit corporations. The analysis shows that science and technology are becoming less disruptive across fields. This applies to both papers (see figure above) and patents (see figure below). For papers, the decrease between 1945 and 2010 ranges from 91.9% for the Social Sciences to 100% for the Physical Sciences; for patents, the decrease between 1980 and 2010 ranges from 78.7% for Computers and Communications to 91.5% for Drugs and Medical

For both papers and patents, the rates of decline are greatest in the earlier parts of the time series, and for patents, they appear to begin to stabilize between 2000 and 2005. For papers, the rate of decline has been more modest in the Life Sciences and Biomedicine and the Physical Sciences since 1980, but the same period has seen the most dramatic and persistent decline in the Social Sciences and Technology. Overall, however, relative to earlier eras, recent papers and patents do less to push science and technology in new directions. 

These findings are furthermore reflected in language; whereas disruptive papers and patent titles use words evocative of creation, discovery and perception (such as “produce”, “determine” or “measure”), consolidating counterparts tend to use more incremental language (such as “improve”, “enhance” or “increase”). Looking more broadly at the content of papers and patents, disruptive ideas are more likely to use new words, or existing words in novel combinations. Once again, an analysis of word diversity echoes the finding that disruptiveness has been declining since 1970 for papers, and since 1990 for patents.

Policy implications for science and technology

The decline in disruptive research underlines concerns about slowing innovation and productivity. While the underlying reasons for this pattern are not yet well understood, the scale of the problem revealed by this analysis should serve as a call to action. For example, scholars may be encouraged to read widely and should be given time to keep up with the rapidly expanding knowledge frontier. Universities may forgot the focus on quantity and more strongly reward research quality, perhaps more fully subsidizing year-long sabbaticals and setting specific provisions for interdisciplinary collaboration. Funding agencies may invest in riskier and longer-term individual awards that support careers, and not simply specific projects, giving scholars the time to explore outside defined parameters, safeguard them from the “publish or perish” culture, and produce truly consequential work. Understanding the decline in disruptive science and technology more fully permits a much-needed rethinking of strategies for organizing the production of science and technology in the future.