I wonder if "publish or perish" may be counterproductive.
It incentivizes people to look for topics that reliably produce papers, it incentivizes grant system to reward people that produce papers, and given that we have a replication crisis on our hands, it seems that quantity beats rigor.
It certainly is counterproductive. We should be incentivising replication, not novelty. If you started giving out grants aimed at replicating results we would speed up scientific progress ten fold.
I always thought that if I ran my own lab, I would have rotation projects be to replicate a recent result from the lab. Then, a lot of the infrastructure and support would still be there, so it would be pretty clear if the fault lay with the experiment. Plus, it would reinforce the notion among trainees that the point of science is to be replicated, and that the hard part of doing something novel is figuring out what to do, not actually doing the work.
Publish methods and results in a database. Every result will be a draw from a distribution. Today only exciting ones get published but it would be better to see the full distribution.
That's good. Replication is valuable for science. Boring is fine. Thinking that boring results aren't useful is exactly how we ended up with the current problems in research. You just need to make sure that you don't give scientists an incentive to redo studies that have been replicated so much that additional replication is useless.
And he's pointing out that your problem is not really so big so as to allow a casual dismissal of the solution. You've reduced the solution to absurdity by creating a scenario that is at least on face easily remediable.
But it's not easily remediable. And that is the point.
Just imagine how that would work in practice. Somebody does an original experiment that gives exciting results. Let's say they get a 1000 Science Points for that. Now somebody replicates that experiment. How many points should they get? Should you get as many points for replicating a 1000 points experiment as replicating a 500 points experiment? Why would you do original experiments at all anymore? Isn't it easier just to replicate original experiments where all of the hard work has already been done?
Awarding points for replication is a nonsense idea.
Well, you wouldn't get 1000 science points for it before it's replicated by others. Once it's replicated enough times by credible people, the points are awarded, and the process is now considered "done".
If someone replicates it after this process and finds different results, it's "new research" again and needs to be replicated again.
I agree, it's kind of like people handing in their "finished" feature with "just the tests" missing. It's not finished. It maybe doesn't even work. Simple as that.
I think we shouldn't even accept papers that haven't been replicated twice by independent teams.
Doesn't really matter WHEN you get the points for your original experiment. My argument stays the same.
Honest researchers doing original research to the best of their abilities. That's how you get good results, and what drives progress. The rest is just bureaucracy. The need for replication will be just another bureaucratic add-on to catch dishonest researchers, and researchers who value prestige more than the truth.
Giving science points to anyone for every replication they do is one idea. Another way to encourage replication without detracting from original research might be to encourage doing one replication. Maybe something like getting masters students to do them as their thesis or final project.
I think the Nobel prize data can be explained quite simply: The Nobel prizes are only awarded in fields that were established early in the 20th century. Because they are long-established, we are seeing diminishing returns as the low-hanging fruit has been plucked. The latter half of the 20th century saw rapid development of new fields such as computer science ... and Nobel prizes are not awarded in these fields.
Another example would be plate tectonics --- incredibly important, only settled in the 1960s, but no Nobel prize awarded because there isn't a Nobel prize for geology/earth sciences.
Physics is super old but for most of that time we didn't have the intellectual tools to do physics well. Once maths, engineering and the scientific method developed enough, the experiments were relatively easy to do (as the OP pointed out, Rutherford was able to do his experiments on his own) and the key results followed.
However they created lots of low hanging fruit. For example I would call relativity a monumental milestone, but the discovery that black holes are possible, the Schwarzschild radius etc were low hanging fruit made possible by Einstein's papers
The article mentions the methodology is unsound but says “it’s the best we can do”. It then discusses implications and conclusions based on unfounded claims.
Prove that the number of Nobel Prizes is a good indicator of scientific process. It doesn’t even sound right when said aloud.
To refine the point:
The number of Nobel Prizes establishes a lower bound on the number of “Nobel Prize worthy discoveries”. That bar is ill-defined and constantly moving. What’s more is that many Nobel Prize worthy discoveries may not be awarded for a very long time. If prizes stopped being given to current discoveries and are all given to discoveries in the 70s and 80s then does that say anything about the rate of scientific progress or even large discoveries? No. We don’t know, with the information presented, if it is from changes in the community. What drives Nobel Prizes being awarded? None of this is seriously looked at. It’s the most important part of the article.
I agree that the metric they use is basically useless.
One of the reasons the Nobel Prize committee tends to skip a decade or two, is that the impact of certain discoveries just cannot be assessed right away.
If a discovery is made, but the test requires technology that takes decades to develop and build, no prize can be awarded yet.
It was of course simpler back in the beginning of the 20th century, when unexplained observations were the test and models that could fit them could be judged right away.
Particle physics, for example, has exhausted the low energy domain and there's nothing more to be found there. New discoveries would require monstrous machinery that takes decades to design, plan, and build.
So to conclude that a decade was less productive just by counting the Nobel Prizes awarded to discoveries made in that decade is flawed. The "lag" just increases with difficulty so to speak.
> The number of Nobel Prizes establishes a lower bound on the number of “Nobel Prize worthy discoveries”.
No, it really doesn't establish anything. They give them out every year in predictable numbers, whether there is a big discovery available, or many, has nothing to do with that. In fact, I'd argue that the number of Nobels given out is independent of scientific progress, real or perceived.
Anecdotally, here's a lot more 'ancillary bloat' in academia now. You can't just buy a microscope, it has to go through a purchasing department. You can't buy your own conference tickets, there's a dedicated buyer. Money trickles down through multiple layers, each with their own beaurocracy. On top of your core science costs, you need to budget for public engagement etc. This has meant that there are more incremental studies being approved (that are closer to patient care), but fewer moonshots, which are harder to demonstrate outcomes.
Could it be that having more scientists not only does not give more great results, but also actually hinders great results? Is the size of the scientific community in itself a problem? Things like bureaucracy, ideological performance, and simply social distance between excellent scientists may play a role, no?
There isn’t some large science community. It’s a field that is hostile to everyone in it. There are no protections for grad students or postdocs. Once through those positions you are a moderately paid staff scientist (if you’re lucky) or in another field. There are so few professorships that it isn’t even worth bothering to list as an option. Anyone going into science thinking they’ll be a professor should stop.
If getting as many as possible and as big as possible grants is incentivised, that isn’t a surprise.
More permanent positions, longer funding (like for 10 years), funding good people, letting them do risky projects without penalising them for failures, and relying less on underpaid PhD students and postdocs would probably lead to more explorations. Nowadays , funding incentives are placed in such way, that incremental discoveries of known/expected things for largest possible amount of money are expected and praised.
this article seems to ignore that the modern scientist in addition to having more complex problems to solve also has a much more complex bureaucracy to navigate to get what they want.
It's also the case that if you get a UK EPSRC grant, more than 50% of that grant goes to university "infrastructure". That infrastructure in computer science consists of a heated room you can work in (with three other people), and an old computer I never used (I rather used my own MacBook Pro). Also admin people, who book flights for you.
It is desirable because once we go into zero sum game, constrained growth, war is the only way to get more resources. With infinite growth you can make your own resources without someone else loosing their resources.
There are still some resources that we are constrained with, like oil, but with science coming up with ways to stop using that resource and having reusable energy sources we can stop wars in places where oil is present.
Then if I can have $2 tomorrow for my $1 invested today because I see there is infinite growth I am willing to put my money into more ventures. If there would be no growth I would stick to my $1 and spend it only on necessities we would stagnate and would not have money to invest in things like renewable energy - which in turn leads to scenario with scarce resources.
Remember that not all people want to live peaceful life having their own farm feeding themselves, there is a lot of human predators taking advantage of others. So to have peaceful society we need growth or at least perceived growth so a lot of people don't have to take someone else piece of pie.
Technology doesn't make resources, it uses resources to produce wealth. Improving technology improves the efficiency with which this can be done. Stopping growth in the technological improvements doesn't mean we go back to zero, we simply get stuck at the current level of efficiency.
Also the premise is unrealistic anyway. Technological improvements will still happen incrementally even with zero funding for research. Especially as big changes occur and needs arrive.
Individual subsistence farming has rarely if ever been a tractable solution, we have nearly always lived and worked as a group and those groups have nearly always required a hierarchy, even a single family is nearly always a hierarchy and in subsistence life style even more so I'd suspect.
>It is desirable because once we go into zero sum game, constrained growth, war is the only way to get more resources.
I think that the question about it being "desired" didn't only ask "why is it viewed as desired?" (that's obvious, who wouldn't just like infinite growth if it was possible and had no side-effects?), but also implied "why is it viewed as desired AND feasible?".
So the above is mostly an "argument from wish" (the "wishful thinking" logical fallacy).
Just because the alternative is war [1] doesn't mean the other alternative is guaranteed.
[1] which is also taken for granted instead of proven, how about the alternative is sustainable living and more equality and cooperation
Think about what growth means: You do something today that you did yesterday and you do it better now than you did then. And generally, you do it better than the person that did it before you. You either do it faster, or better, or cheaper.
Whatever it is you do, the learning effect is a human constant. To say there is no growth is to say humanity does not learn anymore or that something prevents us from applying what we learnt.
It's part of the inherited "millenialism" of the modern world, the idea that progress towards some Star Trek future where everybody gets a free pony is inevitable, the human potential is limitless, resource and physical limits don't matter because "ingenuity" (and "we always found a way in the past", as if the handful of historical examples --which was non-industrial from the dawn of time until 3 centuries ago, and had very limited needs and very limited impact potential -- is a infalible predictor and pertutual guarantee for future outcomes).
The US took this idea and run with it, and also has an ingrained notion of a "manifest destiny" to lead the rest 96% of the world that way. Basically what you get when a culture is founded by persecuted religious nuts :-)
Those are not incompatible. If anything it's the contrary, most of this change is towards mindless hedonism (and greed) in the name of the "Star Trek" future...
No, I think luck has little to do with it, but same for any kind of "guarantee" from nature that we anything close to unlimited potential, we're "bound to" colonise the universe, and so on...
The historical transitions are too few to make any general observation from (as in "we'll always make it" just because we succesfully transitioned 2-3 times).
Then there are the concepts and realities of limited resources (minerals, food production restrictions, etc.), externalities (e.g. pollution), low hanging fruit, diminishing returns (e.g. https://en.wikipedia.org/wiki/Eroom%27s_law), and so on.
Lots of other considerations, e.g. https://en.wikipedia.org/wiki/Fermi_paradox#It_is_the_nature... (even at the moment, on top of shitty developments like climate change and pollution, we already have enough nuclear bombs to take civilization back 1000 years, ways to develop biological attacks, and other such niceties).
Rent seeking requires infinite growth. It is assumed, so the game can go on.
...until it can't. But by then the bonuses have been paid and the losses engineered to be socialized.
We can’t expect it in a long term. However, our understanding of the world around us is rather primitive. There’s so much room for growth in both fundamental and applied sciences that the growth potential is “infinite”.
Well, it depends on how you measure it. If you measure by effort, gravitational waves are way more difficult to detect then to predict, no? So shouldn't the confirmation of their existence count for many times their prediction?
The difficulty of assessing the value of the work being done is indeed great. One way to look at this difficulty is as a significantly higher order phenomenon. What I mean is that if science is the discovery of regularities in nature, technology is the application of scientific knowledge to discover replicable tools, and engineering is the application of technology to solve problems at hand, the last is easiest seen, the one before a bit harder to appreciate as it can look a bit frivolous even at the start, and science is even harder to appreciate (what's the economic value of general relativity?).
And if you want to step one level more, there is mathematics.
> And if you want to step one level more, there is mathematics.
The beauty of mathematics, however, is the fact that great discoveries can be made without spending billions on it.
There's little public backlash for funding a handful of people who quietly scribble on blackboards or humbly request a few hours of time on the university's supercomputer.
Compare that to particle physicists who'd like to have a new super-collider or AI researchers who can easily burn 6-figures worth of compute in a single experiment.
Agreed and that's not the point about mathematics. The point was about ascertaining the impact of a mathematical breakthrough .. which is harder than a scientific one, because the idea of what is a "breakthrough" is usually defined within the domain of mathematics itself. A good example is all the number theoretic work, elliptic curves etc. which underpin much of secure communication today, but started off as "pure" in the sense of "won't find applications". That was easy. Now what's the impact of the breakthrough proof of Fermat's last theorem by Wiles?
Well science is hard, that's nothing new. It seems like it is assumed that a great scientific breakthrough is made overnight by seemingly one person, in reality it's the culmination on decades of previous research and observations made by several people.
What we should be really questioning is whether the current system of academia is making things harder or easier to get this groundbreaking discoveries.
If Einstein was born to day and went to the BSc -> masters -> PhD -> postdoc route that's currently on place would he still come up with the relativity theory? Would it take him more time, maybe less?
This article leads to an obvious question: does Eroom’s Law apply as much to the sciences as a whole as it does to pharmaceutical drug discovery? Are we seeing the same dynamics play out in both cases?
I think this has to be the main driver of it - it's just new, valuable scientific information is getting harder to discover in a number of important fields. It's likely the difficulty and expense of building a working experimental fusion reactor today is relatively much higher than building an experimental fission reactor when they were still being worked on. There's just less (relative) low hanging fruit.
Many colleges and universities have replaced recently retired faculty with adjunct professors instead of tenured positions.
Affirmative action in many STEM-related fields continues to play an important role in the fewer number of tenure appointments that are still being made.
In which case, it's no wonder that in the United States, the finance industry has siphoned off much of the country's best talent.
For the more "passive aggressive" types, there are always positions available at the richly funded defense contracting labs, which in many instances today are giving essentially "no bang" for the taxpayer's buck.
I usually like Collision articles, and agree with the title, but this is a terrible measure for physics.
Horgan's book is a better resource for this sort of thing[0]. Obviously there are many problems with the sociology and incentive systems for physics departments; affirmative action, papers with 1000 "coauthors," infrastructure fees for grants involving giving a grad student a pencil, political witch hunts, difficulties in family formation, herding behavior in subjects, byzantine political games, overcrowded winner take all credit. Frankly you'd have to be kind of a combination masochist moron to want to do it these days outside of the rare person whose career is assured post PhD; masochistic morons probably don't make good scientists.
But nobody talks about the fact that an awful lot of "physics" these days is unfalsifiable piffle. Phenomenology, network theory, cosmology, noodle theory, "quantum computing," black hole physics, neural net fiddlers, nanotechnologists, cosmologists, numskulls babbling about "muh multiverse" and "muh simulation hypothesis" -the world let alone the average physics department has entirely too many of these. Nobody in the physics department can make fun of these cranks for entirely political reasons. And at this point the lunatics outnumber the actual scientists, who, you know, can make predictions that can be checked, rather than generating piffle suited for press releases and late night bong sessions.
I don't understand why affirmative action would be a problem for physics departments. What's the basis of that? As an outsider, I'll say that I see the jobs of a physics department as a) making physics b) making physicists c) teaching physics d) funding making physics. How is affirmative action an "obvious" issue?
This isn't obvious to me either. While I can see that hiring to match predetermined racial quota can result in subpar performance, most of the world does not play the diversity/inclusion game in the way that USA does, and does not seem to have better results.
I've come to see affirmative action's value for the future. Hiring was not free from bias in the past. Even if that's no longer true, the existing lack of physicists in a community means less people from that community will have physics as a goal. Kids rarely dream of attaining roles they don't see. And if kids don't see roles on television or in daily life, that leaves their community. Underrepresentation causes an artificial self-fulfilling prophecy.
It's hard to come up with an unbiased way to correct current problems caused by past bias.
Still, there is a natural experiment. There is no affirmative action in Taiwan or Japan (or Czechia, where I live). In that case, shouldn't teams from those countries exhibit some advantages against American ones?
Or is the effect so small that it gets erased by, say, better facilities or funding?
Assuming that capacity to produce valuable scientific results is uncorrelated with society’s historical prejudice (ie people from historically disadvantaged backgrounds are as likely as any other to produce such results, given they aren’t hampered by historical prejudice), seems more reasonable than assuming that “talent” is reducable to a single normally distributed variable that we have consistently been able to measure in practice.
Because even a reasonable set of first steps towards creating artificial mechanical life forms made out of arbitrary atoms do not exist and probably never will. Drexler wrote a stupid science fiction book. Good marketing for chemists for a while I guess. Remember the congressional hearings back in 2005 on how nanotech was going to be 15% of US GDP by 2015? I do!
It incentivizes people to look for topics that reliably produce papers, it incentivizes grant system to reward people that produce papers, and given that we have a replication crisis on our hands, it seems that quantity beats rigor.