24 November 2020

6. Evaluate the reliability of information.

Almost nothing you know or believe about the world is based on your own experience. Almost everything you know or believe about the world you know on trust.

Consequently, it is up to you to assess the reliability of information presented to you. Here is a list of eight questions you should ask: four regarding the information source and four regarding the information itself.

1: What is the quality of the information channel?

Did the information reach you through a peer-reviewed publication, a monograph by a professor, a textbook, an edited secondary publication, an encyclopaedia, a lecture, a presentation, a face-to-face conversation, a newspaper article, a TV broadcast, a blog, a YouTube video, or a social media post? You must gauge the quality of the information according to the quality of the information channel.

2: Who is the primary source of the information?

Is the primary source a specialist scientist, a generalist expert, a civil servant, a professional in the relevant field (an accountant, physician, banker, or lawyer), a journalist, a teacher (at a university, high school, or elementary school), a friend, a co-worker, a person "in-the-know", a politician, a salesperson? Is the author of the information competent? What are her credentials? What is his record?

3: Is the primary source of the information independent?

Who is paying the primary source’s bills? Taxpayers, a newspaper, a television station, a business, a political party, or an interest group (e.g. the pharmaceutical industry, the tobacco industry, the petroleum industry)? Follow the money.

4: What is the intent of the communication?

To encourage, to enlighten, to inform, to educate, to test, to self-aggrandize, to calm, to convince, to confuse, to mislead, to deceive, to enrage, to panic? Cui bono? Who benefits from the communication?

5: How was the knowledge obtained?

Through original research (e.g. from theory, laboratory experiments, field studies), through records research (a meta-analysis, a literature review, an exploratory review), from anecdotes, from hear-say, by guessing, as folk wisdom?

6: Does the information appear to be accurate and complete?

Is the information current? Has someone pre-selected the information for you? Has contradictory information been considered? Have inconsistencies been addressed? Have alternative interpretations been explored?

(The New York Times boasts: "All the News That's Fit to Print". It is the editors who decide on the fitness of a story. And fitness is defined by information content, entertainment value, and possibly an agenda. Editors not only determine what you read, they also determine what you do not read.)

7: Can the information be independently validated?

Was the communication peer reviewed? Are references cited and available? Are research hypothesis, experimental design, data collection, and data analysis described in enough detail that you could replicate the results, at least in principle?

8: Does the information appear to be unbiased?

Are the results statistically significant? Is the effect size statistically relevant? Are the logic of the argument and the conclusions valid?

Thinking in general makes most people uncomfortable. Judging the reliability of information requires you to think. It is your duty as a citizen to think and be well-informed.

Eight questions, four regarding the information source and four regarding the information itself. It's an easy-enough checklist to keep in mind.