How to Spot Pseudoscience: 5 Hidden Reasons Research Can’t Always Be Trusted
Not all research is created equal and detecting real science from pseudoscience can be quite challenging.
I recently attended a networking event and was presented with an opportunity to buy a noninvasive patch that claimed to cure all illnesses using salts, sugars, and an individual’s internal light waves to recruit stem cells and peptides to repair DNA. When I asked, “Is this treatment supported by rigorous human studies and evidence?” I was told 100% yes. I visited the website, and the real answer is 100% no.
This made me realize that many people lean into research and evidence to support different wellness products and supplements, but do they know the difference between good research and bad research. Do they know about the different kinds of journals? Do they know how to spot real science from pseudoscience? I am willing to bet the answer is no.
So this week is the first part of a two-part series where I am going to break down what you need to know when it comes to research and science. This week we are going to talk about the system, journals, peer review, and bias. Next week, we will get into the individual studies and what to look for when evaluating them.
The Currency of Science: Why Researchers May Prioritize Quantity Over Quality
In order to understand why there is a lot of not-so-great science out there, you have to understand the incentive. Scientific research began as a series of experiments to answer specific questions and to enlighten mankind. However, many of the leading scientists in the world have received honors, awards, and accolades for their published works.
There is a lot of ego and incentive tied to research and publications. In academia, publications are the primary currency. If an academician wants a promotion, tenure, or a grant, they need a long list of published papers.
This creates a high-pressure environment where quantity is often prioritized over quality, leading to practices that compromise the integrity of the final product.
How Academic Incentives Compromise Scientific Integrity
One practice is called “Salami Slicing.” This is when a researcher conducts one large study and slices it into smaller studies to increase their number of publications. While this is not unethical, it can compromise the evidence through statistical manipulation. It is much easier to demonstrate a large statistical difference with a smaller number of study participants.
Unfortunately, unethical practices exist as well. A famous example is the now retracted paper by Andrew Wakefield, which proposed a link between the MMR vaccine and autism. It wasn’t until an investigative report revealed that his study was funded by a law firm preparing to sue vaccine manufacturers, and that Wakefield had falsified the data,1,2 that the truth came out.
Wakefield was neither the first nor the last to commit research misconduct3 and compromise the quality of published science.
The Hierarchy of Scientific Journals: Not All Publications Are Equal
There are typically multiple journals that focus on a specific topic. For example, there are over 20 different journals that publish articles on emergency medicine. However, they are not all high-quality journals that publish good research.
Journals are assigned a score known as an impact factor, which reflects how often articles in that journal are cited. In general, the higher the impact factor, the more reliable the journal, although this can be manipulated.
A journal may choose to publish a controversial, lower-quality article to increase citations. Despite this, impact factor is generally reliable. Journal Citations Reports and Scimago provide information about different journals including impact factors.
It’s also important to know that indexing doesn’t equal validation. Databases such as PubMed, Scopus, ERIC, and PsycINFO function more like libraries. PubMed, for example, focuses on medical journals. Just because something is PubMed indexed does not mean it’s high-quality. It just means it exists.
Implicit vs. Explicit Intentions: Is the Journal Truly Objective?
Every legitimate journal publishes its mission on its website, usually on the “About” page. This is the explicit mission that tells authors who want to publish, and readers who want to subscribe, what to expect in terms content. For example, someone who wants to publish a paper about space exploration technology is not going to submit it to the New England Journal of Medicine, and those who are interested in reading about space exploration are not going to subscribe to the NEJM.
Some journals also carry implicit missions. Certain journals may pursue a predetermined agenda. For example, an alternative medicine journal may have an explicit mission to provide rigorous, unbiased data on alternative medicine practices, but implicitly it refuses to publish any papers that demonstrate certain alternative practices or supplements do not work. This is not providing rigorous, unbiased data. It is creating an echo chamber for a specific ideology.
The best way is to identify this is to examine the content in a journal. If every paper is pushing the same ideology, that journal has an implicit agenda. Legitimate journals LOVE to publish papers with conflicting results because this leads to more studies on that topic, more submissions, and more subscriptions. Think of it as the nerdiest form of the Bravo channel ever: controversy equals money.
Understanding Peer Review Rigor
Every legitimate journal has a rigorous peer review process. Papers are submitted and the journals find 2-3 people who have expertise on that topic to review them. The reviewers go over the paper with a fine toothed comb and provide feedback on every aspect, calling out areas that may be questionable. They also determine if the paper is accepted, rejected, or returned for revision. The authors receive a detailed list from every reviewer explaining why the paper was rejected or revision was requested. I had the honor of being a peer reviewer for several journals, and my comments were at least 1.5 pages long. What never happens is that a paper is submitted and immediately accepted without any request of revision.
Journals that don’t have a strong peer review process are not legitimate journals. They claim to follow a peer review process, but it may just be a rubber stamp. One thing to look for is reviewer transparency. Many journals are now using “Open Peer Review” and publishing the names of the reviewers, which allows readers to look up reviewers and verify their qualifications. Also look at the timeline. Typically when the paper was submitted, revised, and accepted is listed. If all of this happened over a couple days, that is suspicious. This process generally takes at least 6-8 weeks. Lastly, be aware of preprints. If an article comes from bioRxiv or medRxiv, it has not been peer reviewed at all.
Predatory Journals: The Pay to Play Pathway
There are also predatory journals which solely exist to make money. They often charge researchers thousands of dollars to publish their work. These papers undergo minimal peer review, if any. These journals typically have names that are very similar to legitimate journals, and papers that come from them should not be relied upon as credible science.4
Bottom Line
This system is flawed. Academia treats publications as currency, and wherever there is an opportunity for gain, there is an opportunity to game the system. Research misconduct, predatory journals, and publication bias are not rare exceptions. They are well known problems within the science itself.
My hope is that with this background, you now have more tools to evaluate what you read. When you see a study cited to support a wellness product or supplement, you will know to ask: What journal published this? What is its impact factor? Was there a rigorous peer review process? Is this journal indexed and unvetted?
Next week, we will go one level deeper and look at the individual studies themselves, what to look for, and what to skeptical of.
Disclaimer: Even though I’m a doctor, I’m not your doctor—and reading this blog does not establish a doctor–patient relationship. This information is intended for general educational purposes only and should not be taken as personalized medical advice. Always speak with your own healthcare provider before making decisions about your health.
References
- Godlee F, Smith J, Marcovitch H. Wakefield’s article linking MMR vaccine and autism was fraudulent. British Medical Journal Publishing Group; 2011.
- Deer B. How the case against the MMR vaccine was fixed. Bmj. 2011;342
- Gilbert FJ, Denison AR. Research Misconduct. Clinical Radiology. 2003/07/01/ 2003;58(7):499-504. doi:https://doi.org/10.1016/S0009-9260(03)00176-4
- Laine C, Babski D, Bachelet VC, et al. Predatory Journals — What Can We Do to Protect Their Prey? New England Journal of Medicine. 2025;392(3):283-285. doi:doi:10.1056/NEJMe2415937