To some extent, I don’t.

Which is to say that in and around my field (biochemistry), I’m pretty good at sort of “vibe checking”. In practice, this is just a subconscious version of checking that a paper is published in a legit journal, and having a sense for what kind of topics, and language is common. This isn’t useful advice though, because I acquired this skill gradually over many years.

I find it tricky in fields where I am out of element, because I am the kind of person who likes to vet information. Your question about how to identify work as peer reviewed seems simple, but is deceptively complex. The trick is in the word “peer” — who counts as a peer is where the nuance comes in. Going to reputable journals can help, but even prestigious journals aren’t exempt from publishing bullshit (and there are so many junk journals that keeping up even within one field can be hard). There are multiple levels of “peer”, and each is context dependent. For example, the bullshit detector that I’ve developed as a biochemist is most accurate and efficient within my own field, somewhat useful within science more generally, slightly useful in completely unrelated academic fields. I find the trick is in situating myself relative to the thing I’m evaluating, so I can gauge how effective my bullshit detector will be. That’s probably more about reflecting on what I know (and think I know) than it is about the piece of material I’m evaluating.

In most scenarios though, I’m not within a field where my background gives me much help, so that’s where I get lazy and have to rely on things like people’s credentials. One litmus test is to check whether the person actually has a background in what they’re talking about, e.g. if a physicist is chatting shit about biology, or a bioinformatician criticising anthropology, consider what they’re saying with extra caution. That doesn’t mean discount anyone who isn’t staying in their lane, just that it might be worthwhile looking into the topic further (and seeing who else is saying what they are, and what experts from the field are saying too).

As I get deeper into my academic career, I’ve found I’m increasingly checking a person’s credentials to get a vibe check. Like, if they’re at a university, what department are they under? Because a biochemist who is under a physics department is going to have a different angle than one from the medical research side, for example. Seeing where they have worked helps a lot.

But honestly a big part of it is that I have built up loose networks of trust. For example, I’m no statistician, but someone I respect irl referenced a blog of Andrew Gelman’s, which I now consider myself s fan of (https://statmodeling.stat.columbia.edu/). Then from that blog, I ended up becoming a fan of this blog, which tends to be about sociology. Trusting these places doesn’t mean I take them at face value for anything they say, but having that baseline of trust there acts as a sort of first pass filter in areas I’m less familiar with, a place to start if I want to learn about a perspective that I know the rough origin of.

In the context of news, I might start to see a news outlet as trustworthy if I read something good of theirs, like this piece on 3M by ProPublica, which makes me trust other stuff they publish more.

Ultimately though, all of these are just heuristics — imperfect shortcuts for a world that’s too complex for straightforward rules. I’m acutely aware of how little spare brain space I have to check most things, so I have to get lazy and rely on shortcuts like this. In some areas, I’m lucky to have friends I can ask for their opinion, but for most things, I have to accept that I can’t fact check things thoroughly enough to feel comfortable, which means having to try holding a lot of information at arms length and not taking it as fact. That too, takes effort.

However, I got a hell of a lot smarter when I allowed myself to be more uncertain about things, which means sometimes saying “I don’t know what to make of that”, or “I think [thing] might be the case, but I don’t remember where I heard that, so I’m unsure”, or just straight up “I don’t know”. Be wary of simple and neat answers, and get used to sitting with uncertainty (especially in modern science research).

Experience of credibility with a source. If you know a news site is credible, then it’s appropriate to trust at least most things.

If it’s a journal or a blog, then it’s most likely opinion with no real substantial evidence.

Experience of the writer of the source. A lot of official articles will have a small bio about the writer or at least their name so you can research the writer.

Citations. That’s all on that point

Site security. If it’s an unsecured site, then it is not a good source of information

Verbiage. If bias or insulting language is being used, then it’s a bias source which makes it a bad source.

A small thing, and only for “creator” content and people. If they say multiple things, some you know about and some you don’t, then evaluate the stuff you know. If you detect bullshit in the stuff you know, throw it all out.

Someone that lies on one thing is fully untrustworthy.

Clearly doesn’t work when you are wrong.

I dont have an answer other than vet with an authority you trust (be it wikipedia, a teacher a friend, a parent)

But this is not a stupid question. Its probably the most important question when making a decision in this modern techno era to have an answer for yourself

I’m not checking things for peer review, but a lot of bullshit can be filtered out by a simple Google search. If Aunt Brenda posts a major event on Facebook, but it’s not on any news site, she probably fell for a lie.

The discussions here are a bit prosaic, though valid, but on a higher philosophical view you can check Descartes Discourse on the method. It is the basis of all natural sciences and the philosophical foundation of science and rational truth establishment. Maybe grab an explaineer on those ideas.

There are further developments that discuss the sociological proceeds of the scientific community. But the best start point is to always check any statement of truth and fact for four things: controversies, criticisms, corrections and praises. With those four elements you can assert for yourself the credibility of a source’s claims.

It’s peer reviewed if it has the name of a peer-reviewed journal on it. If it’s on arxiv (a pre-print server) it’s not. (Or not yet, or published on several platforms/journals.)

If its peer reviewed then it should go through the experimental setup, the data and accompanying math. They can be evaluated by anyone with enough basic knowledge with math usually being the limiting factor. For example there was this study about animal intelligence that the criteria was if they could recognize themselves in a mirror. Birds and dolphins made the cut but not dogs. My complaint was it was biased to animals where vision was their more primary sense. Now im not an expert in the field but I can still find fault in that way.

The short answer is you don’t. Even in philosophy, a leading model of “truth” is something like “a statement is true if it’s true”. We humans are doomed to be confused and unsure.

There are different standards in different fields of knowledge. Medical science is different than journalism, which is different from history, which is different from public safety.

In general, a given field has sources that publish information with the highest standard of credibility. In many fields, these are peer-reviewed journals. They may be published by large universities (Harvard Law Review, Oxford Review of Economic Policy), by government bodies (e.g. Smithsonian Magazine, NIHR), by professional organizations (eg. JAMA, Annals of Internal Medicine), or operate independently (e.g. The Lancet, Nature).

Media literacy crash course https://youtube.com/watch?v=sPwJ0obJya0&list=PL8dPuuaLjXtM6jSpzb5gMNsx9kdmqBfmY&index=1&pp=iAQB

The main point is to be able to handle uncertainties in a normal basis, the greyness of reality, despite the temptation of blacks and whites of our minds.

For sure it costs a lot. The consideration of the superposition of possible truths and the weight of potential biases is a huge burden without granted full coverage, but allows you to accumulate a landscape of plausibility of things: yes, is not 100% precise and is still built by personal prejudices but, with a systematic acceptance of new bits of information regardless of how comfortable they are, it can grow a mostly reliable understanding of reality with a variable amount of temporary uncertainty on some facts… and you can still convert greys into quasi-b&w once they reach a decent amount of independent evidences, you now, to free a bit your RAM.

PS: Peer review is neither 100% perfect, is just more solid.

Credibility is earned by being consistently credible. A source that posts misleading or false articles can be assumed to not be credible, and I don’t trust them just like I don’t trust people who say stuff that ends up being not credible.

With newer information, concensus between difference sources us a good indicator as well.

What I am far more likely to use to dismiss something is checking out the purpose of the group. If they have a website and their description sounds like a weasel pretending to be a benevolent protector of a hen house then I just ignore them. Anything that sounds pie in the sky, like revolutionizing or disrupting an established industry is probably another Theranos and easily dismissed. If they say anything that sounds like conservative doublespeak, they get ignored.

It seems to be a pretty reliable system even if the occasional thing that is too good to be true slips in because I want it to be true. But having low expectations and recognizing potential being different from the results helpas a lot with being pleasantly surprised when things turn out better than they sounded.

If we can agree that all “news” sites slant to the right or the left. Then you should check out the story at a few of both leaning sites.