The Average Level Of Evidence Of Papers Published In Six Orthopedic Journals

Abstract

Background

An assessment of the average level of evidence (LOE) of clinical research papers in peer reviewed orthopedic surgery journals has not previously been published. In this paper we report on the average LOE of papers published in six of the top English language orthopedic surgery journals.

Methods

Six well established orthopedic journals were identified by using the Google Scholar H5 Journal Index.  We selected 6 of the top ten journals identified by the Index. The six journals that were evaluated for this project were as follows:  The American Journal Of Sports Medicine (AJSM); Clinical Orthopedics and Related Research (CORR); Journal of Bone and Joint Surgery (JBJS); Knee Surgery, Sports, Traumatology, Arthroscopy (KSSTA); Journal of Arthroscopy (Arthros.); and the Journal of Shoulder and Elbow Surgery (JSES). Thirteen consecutive monthly issues (July, 2018 - July, 2019) of each journal were evaluated for the LOE of each clinical paper. The assigned LOE was used if published, or the level of evidence was assigned by the senior author (DAB) if one was not listed. For each journal, the overall mean LOE along with the variance and standard deviation was calculated with all confidence intervals set at  95%. Summary data was statistically analyzed to provide an overall mean LOE for the 6 journals evaluated.

Results 

For AJSM, we evaluated 246 clinical research articles that had an average LOE of 2.96 +/- 0.16. For CORR, we evaluated 155 papers that had an average LOE of 2.83 +/- 0.19.  For JBJS, we evaluated 164 papers that had an average LOE of 3.04 +/- 0.13. For KSSTA, we evaluated 402 papers that had an average LOE of 3.05 +/- 0.16. For Arthros. we evaluated 226 papers that had an average LOE of 3.19 +/- 0.10. For JSES, we evaluated 237 papers that had an average LOE of 3.25 +/- 0.13.

In summary, for the 1425 clinical research papers published in 98 issues of 6 of the top orthopedic journals, we found the average Level Of Evidence to be 3.05 +/- 0.16. 

Conclusions

     The goal of scientific study design is to prove or disprove a hypothesis while minimizing bias and errors.  The gold standard for therapeutic studies has been a double blind, randomized controlled trial (RCT). However, it is not possible or feasible for every clinical study to have a blinded RCT design.  Additionally, clinically relevant and actionable data can come from studies that are not RCTs. In a medical discipline like orthopedics many interventions are invasive, making controlled trials or blinded trials unethical or technically not practical. Our analysis of over 1400 peer reviewed publications in one year in 6 of the top English language orthopedic journals reveals that the average level of evidence deemed worthy of publication is level 3. In terms of the most common study design, the average published clinical study in these orthopedic journals was a case-control study.