Fake news is not a new phenomenon. In fact, deceptive and unethical biomedical reporting has been around for a long time. Take, for example, the now-retracted report in The Lancet by Andrew Wakefield and colleagues linking MMR vaccination with development of inflammatory bowel disease and autism.¹
Given that poor editorial oversight was demonstrated by The Lancet, one of the world's most prestigious medical journals, and may be more readily expected of the many predatory journals springing into existence, practitioners need basic critical appraisal skills to detect weak and misleading studies.
Applying a few simple steps can detect bias in medical studies. The Cochrane Collaboration² highlights the following considerations:
- Are compared groups reasonably similar before the study? Selection bias is diminished with randomization of participants.
- Is the experience of participants different during the study depending upon the arm of the study they are in? Blinding so neither participants nor researchers know how participants are allocated is effective in diminishing performance bias.
- Do outcome assessors know the allocation of participants? Again, blinding is effective in detecting bias so outcome assessors are not influenced by allocation when reporting on a study’s results.
- Are all aspects of the study reported as originally intended? Outcome reporting bias through suppression of "undesirable" results can exaggerate claims of benefit and miss warnings of harm.
A red flag for potential bias in the gold standard of studies, the systematic review, is authorship by a single person. Systematic reviews require many decisions about the validity and relevance of studies and data. Double checking between two or more authors diminishes prejudice and human error.
References:
- Wakefield AJ, Murch SH, Anthony A, Linnell J, Casson DM, Malik M, Berelowitz M, Dhillon AP, Thomson MA, Harvey P, Valentine A, Davies SE, Walker-Smith JA. Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children. Lancet. 1998 Feb 28;351(9103):637-41. Erratum in: Lancet. 2004 Mar 6;363(9411):750. Retraction in: Lancet. 2010 Feb 6;375(9713):445.
- Assessing Risk of Bias in Included Studies. Cochrane Bias Methods Group, Cochrane Collaboration. Belfast. Available from: https://methods.cochrane.org/bias/assessing-risk-bias-included-studies