What about in science?
A basket of indicators all seem to document a trend similar to what we see with technology. Even as the number of scientists and publications rises substantially, we do not appear to be seeing a concomitant rise in new discoveries that supplant older ones. Science is getting harder.
Before diving into these indicators, I want to head off one potential misunderstanding. My claim is that science is getting harder, in some sense, not that science is ending or that we are on the verge of running out of ideas. Instead, the claim is that discoveries of a given “size” are harder to bring about than in the past.
Prior to the 1970s, on average 90% of the time, awards went to papers published in the last twenty years. But by 2015, the ten-year moving average was closer to 50%.
A few points are notable from this exercise. First, physicists seem to think the quantum revolution of the 1910s-1930s was the best era for physics and it’s been broadly downhill since then. That’s certainly consistent with discoveries today being in a sense smaller than the ones of the past, at least for physics.
In contrast, for chemistry and physiology/medicine, the second half of the twentieth century has outperformed the first half. In the Nobel prize data, within the second half of the century, there is no obvious trend up or down for chemistry and medicine. While that’s better than physics, it remains consistent with the notion that science might be getting harder. As we can see in the first figure here, the number of papers and scientists rose substantially between 1950 and 1980, which naively implies that the number of candidates for Nobel-prize winning discoveries should also have risen substantially. If we are selecting the most important discovery from a bigger pool of candidates, we should expect that discovery to be judged more important than discoveries picked from smaller pools. But that doesn’t seem to be the case.
So Nobel prize data is also consistent with the idea that discoveries today aren’t what they used to be. Whereas it used to be quite common for work published in the preceding twenty years to be recognized for a Nobel, that doesn’t happen nearly so much today. That said, an alternative explanation is that the Nobel committee is just trying to work through an enormous backlog of Nobel-worthy work which they want to recognize before the discoverers die. In this explanation, we’ll eventually see just as many awards for the work of today.
But it’s not clear to me this is how the committee is actually thinking: recent work is awarded half the time still if the committee thinks the discovery is sufficiently important. For example, Jennifer Doudna and Emmanuelle Charpentier were awarded a Nobel for their work on CRISP-R in 2020, less than a decade after the main discoveries. And when you look specifically at the work performed in the 1980s, it doesn’t seem particularly notable, relative to work in the 40s, 50s, 60s, and 70s, despite the fact that many more papers were published in that decade.
Still, perhaps the Nobel prize is simply too idiosyncratic for us to learn much from. Next, let’s look at another indicator of big discoveries, one which shouldn’t be biased by the sort of factors peculiar to the Nobel. This is the most top-cited papers in a given field. For example, if we look at the top 0.1% most highly cited papers of all time in a particular field, we could ask how easy is it for a new paper to join their ranks. If that has fallen over time, then that’s further evidence that today’s papers aren’t making the same contributions as yesterday’s.
On the other hand though, we might think it should get harder and harder to climb to the top 0.1%, even if discoveries are not getting smaller. After all, if discoveries are of constant size, earlier works have more time to get citations; it may not be possible for later papers to catch up, even if they are just as good. But there are also some factors that lean in the opposite direction. First, if work is only cited when relevant, then newer work should have an easier time being relevant to newer papers. Since the number of new papers grows over time, that gives one advantage to the new; they can be tailored to a bigger audience, in some sense. Second, the most esteemed papers of all time may actually stop being cited at high rates, because their contributions become part of common knowledge: it is no longer necessary to cite Newton when talking about gravity, or even Watson and Crick when asserting DNA has a double-helix shape.
So let’s proceed with seeing if there has been any change in how easy or hard it is to become a top cited paper, noting that won’t be the last piece of evidence we look at.
(In blue is a related measure, the year-to-year correlation of the rank of top 50 cited papers. In an appendix, they specifically show that this measure is correlated with time alone, so that there is less turnover in more recent years)
If being a top-cited paper is an indicator of large scientific impact, then the above suggests it’s harder to have a big impact than in the past.
Other interpretations are also possible though, such as the factors mentioned earlier. Alternatively, perhaps as fields grow a canon of select papers emerges, and everyone frames their work in relation to this canon, so that earlier work is perpetually cited, but citations don’t accurately capture the “size” of a new discovery. As with the factors peculiar to the Nobel prize, it might be that these two explanations can coexist. But in case there is something strange about how earlier work is necessarily canonized, let’s now turn to some indicators that cover science more broadly.
So let’s turn to a metric that isn’t based on the assessment of the scientists themselves. Rather than looking at the size of discoveries, we could instead try to chart the topics covered by a field over time. If a field is steadily spreading into new topics (from electrons to quarks to strings, for example), that suggests a field is learning new things and pushing out its frontier. On the other hand, if a field remains stuck on the same set of things (from strings to strings to strings, for example) that might be indicative that the field is struggling to make progress.
Now that Milojević has a way to define topics in a field, she then goes about counting how many distinct topics are mentioned in the titles of papers published in a given year. Analogous to the number of papers published per person, Milojević looks at the number of unique topics for every set of 10,000. In other words, the algorithm reads random paper titles until it reaches 10,000 topics, and then counts how many topics from this set of 10,000 are unique.
Through the twentieth century, there was a general rise in the number of unique topics studied in a given sample of scientific titles. Over 1935-1975 this rise was a bumpy one, but it looks like we mostly reverted to trend, so the overall rate of change was steady over a long horizon. But sometime since the 1970s, this upward trend has very gradually slowed to a stop, and even began to reverse slightly. If counting topics is a good way to measure the successful growth rate of a field, then this indicates fields are having a harder time growing today than in the past.
Another approach we could take is to compare the citations received by papers over time. If older papers made bigger discoveries than younger ones, then we might expect them to hold on longer and be more highly cited than new papers. One simple way to assess this is to look at the share of citations made in each year to new papers. A simple measure of this is the Price index (named for Derek De Solla Price, not the cost of a good), which computes the share of citations made to papers published in the last 5 years (or 10 years in some variants).
I think this figure is best understood as describing two periods. From 1900-1955 there was a general increase in the share of citations made to recent work, interrupted by the two world wars. Each world war imposed big disruptions on the production of new scientific work (see the first figure in this post), which had the secondary effect of reducing the share of citations to recent work. But since 1955, the share of citations made to new work has fallen dramatically, by an amount comparable to the distortions caused by the world wars (though spread out over many decades).
Cui, Wu, and Evans (2022)
This looks pretty alarming, but there are explanations besides a sharp decline in size of discoveries. Whenever the share of something goes down, there are two possible causes: it could be the numerator goes down and/or the denominator goes up. And in this case, it’s mostly the latter. The raw number of citations to recent work doesn’t actually seem to have fallen by very much, at least, according to Larivière, Archambault, and Gingras (2007). But the total number of citations papers make has gone way up, and most of that increase has been citations to older work.
So the real question is less “why have researchers stopped citing new work” and more “why are researchers citing old work at such a high rate.” One explanation is that older work contains the bigger discoveries, and we’re still living in their shadow. But another explantion, put forward by Cui, Wu, and Evans, is simply that the scientific labor force is aging and older scientists are more familiar with older papers. Older scientists might push up the share of citations to older paper either by citing them in their own work, or by insisting on them being cited in other people’s work when they serve as peer reviewers. This age dynamic could also be a factor in the persistence of top-cited papers of the past. But whether this is about the size of discoveries or an aging scientific labor force’s preferences (and the two explanations are not mutually exclusive either), the more distant past is increasingly influential in contemporary science.
So let’s close by looking at one more data source, which is somewhat immune to some of the strategic citation factors unique to academia.
This figure echoes what we see in the academic price index: citations to recent work have become increasingly less common. Unlike the academic work though, this is not entirely a story of rising citations to older work and steady citations to new work. Instead, we are actually seeing a decline in the number of citations to recent work. Among the subset of patents that cite academic work, the average number of citations to papers published in five years prior to the patent’s filing date dropped from about 4 around 2000 to under 3.5 by 2015.
Stepping back, I’m claiming that science is getting harder, in the sense that it is increasingly challenging to make discoveries that have comparable impact to the ones in the past. Diverse groups – the Nobel nominators, contemporary surveyed scientists, academics, and inventors – all seem to have an increasing preference for the work of the past, relative to the present. And looking at growth in the number of topics covered by scientists also suggests it has become harder to make forward progress. To close, I’ll add two more arguments.
Each of these pieces of evidence has holes in it. But I think they are not the same holes. Stack them all up, and I think you get an argument that can begin to hold water.
A few extra notes:
In anappendix , I briefly list some additional possible explanations for why science might be harder. A good overview of some of the newer metrics for quantifying science, which helped me in drafting this post, isWu et al. (2021) . I have turned off comments on substack for this post, but if you would like to comment or discuss it, I invite you to do so at the newProgress Forum , which I think is a better permanent home for comments.This link goes to this post on progress forum.
If you want to chat about this post or innovation in generally, let’s grab a virtual coffee. Send me an email at mattclancy at hey dot com and we’ll put something in the calendar.