Also: how do you identify a work as peer reviewed?
I’m not checking things for peer review, but a lot of bullshit can be filtered out by a simple Google search. If Aunt Brenda posts a major event on Facebook, but it’s not on any news site, she probably fell for a lie.
A small thing, and only for “creator” content and people. If they say multiple things, some you know about and some you don’t, then evaluate the stuff you know. If you detect bullshit in the stuff you know, throw it all out.
Someone that lies on one thing is fully untrustworthy.
Clearly doesn’t work when you are wrong.
No person nor source gets it right all the time, I like your idea as an evaluative technique but I think the assumption that being incorrect here is necessarily because of lying might mean discounting a lot of sources/creators who are otherwise reputable. I’d look at it more like degrees of doubt cast over everything else they say where you don’t have the expertise to evaluate the accuracy. Much like a driver’s licence, you get dinged enough times for more and more infractions and eventually you lose your license. If they keep continually getting things wrong where it’s something you know something about eventually you can probably discount them on anything else as well, but if it’s just once or twice, especially where they’re not egregiously wrong, some benefit of the doubt could be beneficial to all concerned. Better I would have thought to take what feels like their salient points on the content they produce on topics where you aren’t knowledgeable and check if other people are claiming anything similar and where something is verifiable, try to verify. Of course theoretically you should do that all the time but in practice at least each time you know someone is wrong about something it’s an indication that for them specifically further checking is required.
It’s peer reviewed if it has the name of a peer-reviewed journal on it. If it’s on arxiv (a pre-print server) it’s not. (Or not yet, or published on several platforms/journals.)
It’s peer reviewed if it has the name of a peer-reviewed journal on it.
Where do journals indicate that they are?
A lot of them will have a front matter, a Wikipedia article. Be cited a lot. And you’ll find them in a university library. You might even have access to a library’s catalogue without being a student or member. Being peer-reviewed will be in the description.
And the bogus “journals” are kind of well-known. I think you’ll find information with a simple Google search.
For example: https://en.wikipedia.org/wiki/List_of_scientific_journals
“peer-reviewed” is always within the first sentences if you click on something.
https://en.wikipedia.org/wiki/Wikipedia:Identifying_reliable_sources_(medicine)#Biomedical_journals is a good guide for Biomed
I dont have an answer other than vet with an authority you trust (be it wikipedia, a teacher a friend, a parent)
But this is not a stupid question. Its probably the most important question when making a decision in this modern techno era to have an answer for yourself
If its peer reviewed then it should go through the experimental setup, the data and accompanying math. They can be evaluated by anyone with enough basic knowledge with math usually being the limiting factor. For example there was this study about animal intelligence that the criteria was if they could recognize themselves in a mirror. Birds and dolphins made the cut but not dogs. My complaint was it was biased to animals where vision was their more primary sense. Now im not an expert in the field but I can still find fault in that way.
Though, that’s not peer review. What you’re describing is reproducibility. And that’s the very minimum to qualify as science. If it doesn’t describe the experiment well enough so an expert can follow it… It’s not even proper science.
Peer review means, several expert in that domain already took some time to go through it and point out flaws, comment on the methodology and gave a recommendation to either publish it or fix mistakes. It’s not the ability to do it, but that it actually already happened. And it has to be other researchers from the same field.
And there is even another possible step after that, if an independent other research group decides to reproduce the experiment and confirm and verify the results.
I know what peer review is, its just that peer reviewed things also tend to be scientific studies. I mean I know there are studies of studies and such.
Fair enough. Maybe we had a different understanding of OP’s question. I took it to mean, how can I find out a given article/paper has been reviewed… And that’s not done by looking if it looks scientific, but if the review process has happened.
Has nothing to do with OPs question. You missed the very first sentence to the comment your first responded to
Yeah, I pointed out my reasoning in the other comment to my reply. Sure, if it’s proper peer reviewed since, it’ll follow the process. But that doesn’t answer OP’s question. I agree, however. If it’s proper science, it’s proper science. I just wanted to stick with the question at hand. And there is no causal relationship between peer-review and reproducibility, other than that it’s both part of science. So I got mislead by the … if … then … phrasing.
Your reasoning doesn’t matter if it’s being applied to the wrong problem.
This is not about OP.
Sure. I don’t want to argue. I took it as that, since it was a direct reply to a specific question. And i think my short outline of what the word means is mostly correct.
if its peer reviewed.
You kinda glossed right over that didnt you? Maybe an edit is in your future?
Yeah, but the question was: how does someone find out something is peer reviewed? And phrasing it like this is silly… It’s peer reviewed if it’s peer reviewed… That’s a tautology. Sure it’s true. But it doesn’t mean anything. And if you take the implication the other way round (as I did), it’s wrong. That’s what I pointed out. Minus the tautology part.
Okay Ojay okay. Help me out. Why are you claiming that this is the question?
“The question was: how does someone find out something is peer reviewed”
Please literally show where you see this being asked. It was not asked in the top comment, nor is it necessary to ask. I don’t understand why you feel it is silly or unnecessary as it is very clearly used for a specific purpose when i read the top comment.
Again you are wrong. It was not the question of the top comment you responded to. That was a follow up question that is irrelevant because the comment you that started this discussion cleanly clearly and unambiguously removed any need to discuss how to find out if something is peer reviewed by the first words they started the comment with.If it is peer reviewed…And they gave an example of something that you could do to further verify a peer reviewed paper. You can replicate the experiment and get the same results but then offered an example where there might be a problem with only reproducing the results, to them anywayThat’d be the body text of the post:
Also: how do you identify a work as peer reviewed?
Then HubertManne directly replied to that: “If its peer reviewed then …”
Then I replied saying, everything after the “then” (the main text of the comment) has nothing to do with peer review but is a different concept. So no one gets the impression you can make the judgement the other way around: If it’s doing that, then it’s peer-reviewed. Because that’d be wrong.
And then we started having this lengthy discussion. Do you concur? Or are we having some technical difficulties, and we’re somehow seeing a different post/comment tree?
Okay I’m done with this you are just being obstinate or intentionlly trolling me
The short answer is you don’t. Even in philosophy, a leading model of “truth” is something like “a statement is true if it’s true”. We humans are doomed to be confused and unsure.
Experience of credibility with a source. If you know a news site is credible, then it’s appropriate to trust at least most things.
If it’s a journal or a blog, then it’s most likely opinion with no real substantial evidence.
Experience of the writer of the source. A lot of official articles will have a small bio about the writer or at least their name so you can research the writer.
Citations. That’s all on that point
Site security. If it’s an unsecured site, then it is not a good source of information
Verbiage. If bias or insulting language is being used, then it’s a bias source which makes it a bad source.
The discussions here are a bit prosaic, though valid, but on a higher philosophical view you can check Descartes Discourse on the method. It is the basis of all natural sciences and the philosophical foundation of science and rational truth establishment. Maybe grab an explaineer on those ideas.
There are further developments that discuss the sociological proceeds of the scientific community. But the best start point is to always check any statement of truth and fact for four things: controversies, criticisms, corrections and praises. With those four elements you can assert for yourself the credibility of a source’s claims.
To some extent, I don’t.
Which is to say that in and around my field (biochemistry), I’m pretty good at sort of “vibe checking”. In practice, this is just a subconscious version of checking that a paper is published in a legit journal, and having a sense for what kind of topics, and language is common. This isn’t useful advice though, because I acquired this skill gradually over many years.
I find it tricky in fields where I am out of element, because I am the kind of person who likes to vet information. Your question about how to identify work as peer reviewed seems simple, but is deceptively complex. The trick is in the word “peer” — who counts as a peer is where the nuance comes in. Going to reputable journals can help, but even prestigious journals aren’t exempt from publishing bullshit (and there are so many junk journals that keeping up even within one field can be hard). There are multiple levels of “peer”, and each is context dependent. For example, the bullshit detector that I’ve developed as a biochemist is most accurate and efficient within my own field, somewhat useful within science more generally, slightly useful in completely unrelated academic fields. I find the trick is in situating myself relative to the thing I’m evaluating, so I can gauge how effective my bullshit detector will be. That’s probably more about reflecting on what I know (and think I know) than it is about the piece of material I’m evaluating.
In most scenarios though, I’m not within a field where my background gives me much help, so that’s where I get lazy and have to rely on things like people’s credentials. One litmus test is to check whether the person actually has a background in what they’re talking about, e.g. if a physicist is chatting shit about biology, or a bioinformatician criticising anthropology, consider what they’re saying with extra caution. That doesn’t mean discount anyone who isn’t staying in their lane, just that it might be worthwhile looking into the topic further (and seeing who else is saying what they are, and what experts from the field are saying too).
As I get deeper into my academic career, I’ve found I’m increasingly checking a person’s credentials to get a vibe check. Like, if they’re at a university, what department are they under? Because a biochemist who is under a physics department is going to have a different angle than one from the medical research side, for example. Seeing where they have worked helps a lot.
But honestly a big part of it is that I have built up loose networks of trust. For example, I’m no statistician, but someone I respect irl referenced a blog of Andrew Gelman’s, which I now consider myself s fan of (https://statmodeling.stat.columbia.edu/). Then from that blog, I ended up becoming a fan of this blog, which tends to be about sociology. Trusting these places doesn’t mean I take them at face value for anything they say, but having that baseline of trust there acts as a sort of first pass filter in areas I’m less familiar with, a place to start if I want to learn about a perspective that I know the rough origin of.
In the context of news, I might start to see a news outlet as trustworthy if I read something good of theirs, like this piece on 3M by ProPublica, which makes me trust other stuff they publish more.
Ultimately though, all of these are just heuristics — imperfect shortcuts for a world that’s too complex for straightforward rules. I’m acutely aware of how little spare brain space I have to check most things, so I have to get lazy and rely on shortcuts like this. In some areas, I’m lucky to have friends I can ask for their opinion, but for most things, I have to accept that I can’t fact check things thoroughly enough to feel comfortable, which means having to try holding a lot of information at arms length and not taking it as fact. That too, takes effort.
However, I got a hell of a lot smarter when I allowed myself to be more uncertain about things, which means sometimes saying “I don’t know what to make of that”, or “I think [thing] might be the case, but I don’t remember where I heard that, so I’m unsure”, or just straight up “I don’t know”. Be wary of simple and neat answers, and get used to sitting with uncertainty (especially in modern science research).
Thanks for the extensive response! I appreciate the perspective, particularly the nuances on peer review, and the grounded conclusion.
The main point is to be able to handle uncertainties in a normal basis, the greyness of reality, despite the temptation of blacks and whites of our minds.
For sure it costs a lot. The consideration of the superposition of possible truths and the weight of potential biases is a huge burden without granted full coverage, but allows you to accumulate a landscape of plausibility of things: yes, is not 100% precise and is still built by personal prejudices but, with a systematic acceptance of new bits of information regardless of how comfortable they are, it can grow a mostly reliable understanding of reality with a variable amount of temporary uncertainty on some facts… and you can still convert greys into quasi-b&w once they reach a decent amount of independent evidences, you now, to free a bit your RAM.
PS: Peer review is neither 100% perfect, is just more solid.
Media literacy crash course https://youtube.com/watch?v=sPwJ0obJya0&list=PL8dPuuaLjXtM6jSpzb5gMNsx9kdmqBfmY&index=1&pp=iAQB
There are different standards in different fields of knowledge. Medical science is different than journalism, which is different from history, which is different from public safety.
In general, a given field has sources that publish information with the highest standard of credibility. In many fields, these are peer-reviewed journals. They may be published by large universities (Harvard Law Review, Oxford Review of Economic Policy), by government bodies (e.g. Smithsonian Magazine, NIHR), by professional organizations (eg. JAMA, Annals of Internal Medicine), or operate independently (e.g. The Lancet, Nature).
Credibility is earned by being consistently credible. A source that posts misleading or false articles can be assumed to not be credible, and I don’t trust them just like I don’t trust people who say stuff that ends up being not credible.
With newer information, concensus between difference sources us a good indicator as well.
What I am far more likely to use to dismiss something is checking out the purpose of the group. If they have a website and their description sounds like a weasel pretending to be a benevolent protector of a hen house then I just ignore them. Anything that sounds pie in the sky, like revolutionizing or disrupting an established industry is probably another Theranos and easily dismissed. If they say anything that sounds like conservative doublespeak, they get ignored.
It seems to be a pretty reliable system even if the occasional thing that is too good to be true slips in because I want it to be true. But having low expectations and recognizing potential being different from the results helpas a lot with being pleasantly surprised when things turn out better than they sounded.
If we can agree that all “news” sites slant to the right or the left. Then you should check out the story at a few of both leaning sites.
