I like the sentiment, but there are non-peer reviewed papers that are real science. Politics and funding are real things, and there is a bit of gatekeeping here, which isn’t really good IMHO.
Also, reproducibility is a sticky subject, especially with immoral experiments (which can still be the product of science, however unsavory), or experiments for which there are only one apparatus in the world (e.g., some particle physics).
The things you’re describing are not science. This might seem nit picky but the scientific method as we know it today require that peer review and require methods of reproduction. Whether you can reproduce results is a different story.
The entire difference between research and science is whether or not you engage in the process of peer review and review often requires method of replication. So you usually can’t have one without the other. If you aren’t trying to have your paper reviewed by your peers, that’s fine, but that isn’t science.
To address the gatekeeping, I get it. We shouldn’t be using the word to demean people who do valuable research but don’t strictly engage in the scientific process. That’s really not important to do. However we should all be interested in preventing the scientific process from being muddied to include every R&D process under the sun. That’s all research, not science, and we call them separate things for a reason.
I think the word you’re looking for is merit, publication which are cited and peer reviewed hold much more merit than those who don’t.
Science is a rigorous, systematic endeavor that builds and organizes knowledge in the form of testable explanations and predictions about the world. 1
Nothing in this quote requires external publication. Following the scientific method, publishing, peer reviewing and reproduction can all happen internally in organisation using independent teams. Those private publications hold but a fraction of the merit of publications in recognised journals, but are science nonetheless.
Oh yeah strictly speaking if you follow the scientific method you are doing “science” however what the twitter thread is getting at and what I’m getting at is that science without the scientific process isn’t the same thing. Typically in a professional setting we just call that research.
The scientific process contains the scientific methods but there is an aspect of connection to the scientific community. I’d argue that if you’re using a company to build and develop a working base of knowledge through the scientific method, you’re failing at the building and organizing knowledge part of that science definition by not sharing what you know.
For sure, and calling Elon a twat would be an insult to twats out there. But saying “if it’s not published it’s not science” to one of the greatest grifters while having to explain the nuance of what you tweeted is a big L in my book.
I don’t particularly agree. Publishing is a tricky thing in the private sector, and we’ve seen a lot of scientific suppression by companies. Peer review literally requires the field to assess your work, and doesn’t end with the publication, but is a process that continues forever. Reproduction is a major issue, especially in fields proximal to mine (neuroscience , Medicine and psychology) and the whole process of open science with this type of review process makes it much easier to create papers that are reproducible.
The external influence is basically a given to produce science that holds up.
I agree though, we can argue open science is much better and more reliable. We can argue privatly conducting a study and doing all the steps that would be conducted by the academic community within one organisation leads to more biased and less reliable results. But it’s still science by its very definition, I’d even argue denying that is a bit disrespectful to all scientists doing so.
This might seem nit picky but the scientific method as we know it today require that peer review and require methods of reproduction. Whether you can reproduce results is a different story.
Oooh, are we about to have a discussion on whether large portions of the soft sciences across the past several decades fail to be “real” science due to the reproducibility crisis?
I agree with you last point, and I really, really want to with the first.
Sometimes science feels more like an art, for chemistry at least. I suppose the counter-point to this is: if you provide sufficient detail to reproduce but your results are still difficult to reproduce reliably by others, then your process wasn’t very robust and should have undergone more development before publishing. Those details may be so minor that you don’t even realize that you overlooked something.
Yup, that poor guy that proved that vaccines cause autism gets unfairly maligned for his efforts; obviously his methodology was unethical but you can’t argue the findings.
I like the sentiment, but there are non-peer reviewed papers that are real science. Politics and funding are real things, and there is a bit of gatekeeping here, which isn’t really good IMHO.
Also, reproducibility is a sticky subject, especially with immoral experiments (which can still be the product of science, however unsavory), or experiments for which there are only one apparatus in the world (e.g., some particle physics).
The things you’re describing are not science. This might seem nit picky but the scientific method as we know it today require that peer review and require methods of reproduction. Whether you can reproduce results is a different story.
The entire difference between research and science is whether or not you engage in the process of peer review and review often requires method of replication. So you usually can’t have one without the other. If you aren’t trying to have your paper reviewed by your peers, that’s fine, but that isn’t science.
To address the gatekeeping, I get it. We shouldn’t be using the word to demean people who do valuable research but don’t strictly engage in the scientific process. That’s really not important to do. However we should all be interested in preventing the scientific process from being muddied to include every R&D process under the sun. That’s all research, not science, and we call them separate things for a reason.
I think the word you’re looking for is merit, publication which are cited and peer reviewed hold much more merit than those who don’t.
Nothing in this quote requires external publication. Following the scientific method, publishing, peer reviewing and reproduction can all happen internally in organisation using independent teams. Those private publications hold but a fraction of the merit of publications in recognised journals, but are science nonetheless.
Oh yeah strictly speaking if you follow the scientific method you are doing “science” however what the twitter thread is getting at and what I’m getting at is that science without the scientific process isn’t the same thing. Typically in a professional setting we just call that research.
The scientific process contains the scientific methods but there is an aspect of connection to the scientific community. I’d argue that if you’re using a company to build and develop a working base of knowledge through the scientific method, you’re failing at the building and organizing knowledge part of that science definition by not sharing what you know.
For sure, and calling Elon a twat would be an insult to twats out there. But saying “if it’s not published it’s not science” to one of the greatest grifters while having to explain the nuance of what you tweeted is a big L in my book.
Much more measured take than “if it’s not published, then it definitely is not science.”
I don’t particularly agree. Publishing is a tricky thing in the private sector, and we’ve seen a lot of scientific suppression by companies. Peer review literally requires the field to assess your work, and doesn’t end with the publication, but is a process that continues forever. Reproduction is a major issue, especially in fields proximal to mine (neuroscience , Medicine and psychology) and the whole process of open science with this type of review process makes it much easier to create papers that are reproducible.
The external influence is basically a given to produce science that holds up.
I agree though, we can argue open science is much better and more reliable. We can argue privatly conducting a study and doing all the steps that would be conducted by the academic community within one organisation leads to more biased and less reliable results. But it’s still science by its very definition, I’d even argue denying that is a bit disrespectful to all scientists doing so.
Oooh, are we about to have a discussion on whether large portions of the soft sciences across the past several decades fail to be “real” science due to the reproducibility crisis?
Counterpoint: the scientific method is much simpler than you described.
The rest are details of the above or elitism.
I think the sticking point is this: if people can’t reproduce it then you missed writing down an important detail and therefore didn’t finish step 3.
The elitism is thinking peer review suffices for reproducibility.
I agree with you last point, and I really, really want to with the first.
Sometimes science feels more like an art, for chemistry at least. I suppose the counter-point to this is: if you provide sufficient detail to reproduce but your results are still difficult to reproduce reliably by others, then your process wasn’t very robust and should have undergone more development before publishing. Those details may be so minor that you don’t even realize that you overlooked something.
I mean that makes sense. I guess it would be fairer to say that enough should be written down its still usable in tracking down what is missing.
Imo both in the twitter are stupid. Like, no way engaging with musk could go well.
Yup, that poor guy that proved that vaccines cause autism gets unfairly maligned for his efforts; obviously his methodology was unethical but you can’t argue the findings.