Yea … this is a good ending though … for those paying attention and keen to learn something about science. It was collaboration and multiple perspectives and expertises that got to the truth … it was a mystery with a non-obvious answer that required hard to predict insight and guess work that was all wrapped up by multiple people testing their and others theories. You could probably wrap the whole episode up into a case study on what science actually is.
I also can’t help but notice that the whole episode happened before the initial paper even made it through peer review, which I think raises some reasonable questions about the current bias of science, which leans very heavily on the value of peer review and little on the value of replication studies (with LK99 being unique in how much replication work was easily feasible and attractive).
In my opinion, many fields would benefit from actually committing and rewarding good replication studies, which can and should function as peer review or critiques of the original papers/discoveries too.
Maybe all of those PhD students would have their time better spent on this task than pretending, as if often the case, they’ve done some original work on an important theory that’s found something “for the first time”.
Maybe all of those PhD students would have their time better spent on this task than pretending, as if often the case, they’ve done some original work on an important theory that’s found something “for the first time”.
I mean I’m personally biased as a PhD student myself, but I think this is a great idea. I made the core of my project to basically take a picture of a phenomenon that has been inferred from spectroscopy but not observed directly. So verification, not exactly replication, but same idea. Turns out that doing something like this is very hard and makes a worthy PhD project. (I haven’t managed it yet, and am starting to wonder if my eventual paper might actually end up being in support of the null hypothesis…)
But I’m also not looking to go into academia after I graduate, so I’m not to worried about trying for something high impact or anything like that. I think for someone angling to be a professor the idea of a replication or verification project may be a harder sell, which is largely down to the culture of academia and how universities do their hiring of post-docs and such. I mean, even in this case more people are still going to be familiar with the names of Lee and Kim than any of the researchers who put in work on replication studies (can you name any of them without checking the article?).
tl;dr definitely a worthy goal and replications should absolutely be encouraged, but it’s going to take a while to change the whole academic culture to reinforce that they’re valuable contributions.
And yea, the moment I thought of the idea I had one of those rare feelings of thinking it obviously made sense, at least for the current system we have, where a PhD student is, often, at the beginning, not really qualified to be doing actual research, but the whole system is historically premised on that notion and kind of twists itself into keeping up that appearance (depending on where you are), which I think has plenty of negative knock on effects on the quality of science and researchers. Combining learning how to do research in science with the task of doing that research efficiently (ie publish or perish) is tricky and can get wonky.
So … why not use replication, which the system, IMO, surely needs more of, as a way of teaching research while also doing some form of research that happens to be along slightly better trodden grounds while also emphasising, in a way better than a lot of attempts at “original research” IMO, the true essential process of science.
I’ve got a feeling that such PhD work would actually produce better scientists (depending on the field and location).
Hope your project goes well! Especially with the negative replication/verification result! I’ve heard stories about how that can be a pain to publish, depending on the details of course. Good luck!!
I agree completely, especially about the negative knock-on effects on the quality of science overall. Making replications worthwhile for researchers to spend time and money on is certainly going to be a challenge that the institution of academia will need to figure out sooner or later (fingers crossed for sooner, but realistically probably later).
Good luck with your PhD too! I hope it’s going well so far!
Yea … this is a good ending though … for those paying attention and keen to learn something about science. It was collaboration and multiple perspectives and expertises that got to the truth … it was a mystery with a non-obvious answer that required hard to predict insight and guess work that was all wrapped up by multiple people testing their and others theories. You could probably wrap the whole episode up into a case study on what science actually is.
I also can’t help but notice that the whole episode happened before the initial paper even made it through peer review, which I think raises some reasonable questions about the current bias of science, which leans very heavily on the value of peer review and little on the value of replication studies (with LK99 being unique in how much replication work was easily feasible and attractive).
In my opinion, many fields would benefit from actually committing and rewarding good replication studies, which can and should function as peer review or critiques of the original papers/discoveries too.
Maybe all of those PhD students would have their time better spent on this task than pretending, as if often the case, they’ve done some original work on an important theory that’s found something “for the first time”.
I mean I’m personally biased as a PhD student myself, but I think this is a great idea. I made the core of my project to basically take a picture of a phenomenon that has been inferred from spectroscopy but not observed directly. So verification, not exactly replication, but same idea. Turns out that doing something like this is very hard and makes a worthy PhD project. (I haven’t managed it yet, and am starting to wonder if my eventual paper might actually end up being in support of the null hypothesis…)
But I’m also not looking to go into academia after I graduate, so I’m not to worried about trying for something high impact or anything like that. I think for someone angling to be a professor the idea of a replication or verification project may be a harder sell, which is largely down to the culture of academia and how universities do their hiring of post-docs and such. I mean, even in this case more people are still going to be familiar with the names of Lee and Kim than any of the researchers who put in work on replication studies (can you name any of them without checking the article?).
tl;dr definitely a worthy goal and replications should absolutely be encouraged, but it’s going to take a while to change the whole academic culture to reinforce that they’re valuable contributions.
Also a PhD student! Also not going into academia.
And yea, the moment I thought of the idea I had one of those rare feelings of thinking it obviously made sense, at least for the current system we have, where a PhD student is, often, at the beginning, not really qualified to be doing actual research, but the whole system is historically premised on that notion and kind of twists itself into keeping up that appearance (depending on where you are), which I think has plenty of negative knock on effects on the quality of science and researchers. Combining learning how to do research in science with the task of doing that research efficiently (ie publish or perish) is tricky and can get wonky.
So … why not use replication, which the system, IMO, surely needs more of, as a way of teaching research while also doing some form of research that happens to be along slightly better trodden grounds while also emphasising, in a way better than a lot of attempts at “original research” IMO, the true essential process of science.
I’ve got a feeling that such PhD work would actually produce better scientists (depending on the field and location).
Hope your project goes well! Especially with the negative replication/verification result! I’ve heard stories about how that can be a pain to publish, depending on the details of course. Good luck!!
I agree completely, especially about the negative knock-on effects on the quality of science overall. Making replications worthwhile for researchers to spend time and money on is certainly going to be a challenge that the institution of academia will need to figure out sooner or later (fingers crossed for sooner, but realistically probably later).
Good luck with your PhD too! I hope it’s going well so far!