Also, not just the paper itself. You better make all the data generated during the study available too. That way it can be used to verify the validity of the conclusions, and also used for other studies.
This is so important. It’s astounishing how often I come across papers with results that are not reproducable because they either don’t publish their code or their data sets or neither.
These are effectively useless.
Even more papers where the authors p-hacked the data toward a positive result, or made grandiose conclusions that the data barely suggests, or, specifically in biology, which I like to read, the sample range was so flawed (i.e. too small, bias based on gender, ethnicity, preexisting conditions and life factors, all the subjects are cousins, etc) that it’s completely invalid and any result is way more likely to be due to the nonrandom selection of subjects. These things are how we get shitty invalid papers like that one that linked vaccination to autism, or literally any paper that tries linking race/ethnicity to stuff like crime rate or lack of empathy (I’ve not seen a single paper in that vein that didn’t get absolutely trashed on peer review, and is honestly just written as an excuse to be racist).
Also, not just the paper itself. You better make all the data generated during the study available too. That way it can be used to verify the validity of the conclusions, and also used for other studies.
This is so important. It’s astounishing how often I come across papers with results that are not reproducable because they either don’t publish their code or their data sets or neither. These are effectively useless.
Even more papers where the authors p-hacked the data toward a positive result, or made grandiose conclusions that the data barely suggests, or, specifically in biology, which I like to read, the sample range was so flawed (i.e. too small, bias based on gender, ethnicity, preexisting conditions and life factors, all the subjects are cousins, etc) that it’s completely invalid and any result is way more likely to be due to the nonrandom selection of subjects. These things are how we get shitty invalid papers like that one that linked vaccination to autism, or literally any paper that tries linking race/ethnicity to stuff like crime rate or lack of empathy (I’ve not seen a single paper in that vein that didn’t get absolutely trashed on peer review, and is honestly just written as an excuse to be racist).