Advertisement

The Fragility of Statistical Findings in Distal Radius Fractures: A Systematic Review of Randomized Controlled Trials

      Abstract

      Purpose

      Probability analysis with reporting of P values is often used to determine the statistical significance of study findings in the orthopedic literature. The fragility index (FI) has been used to evaluate the robustness of a significant result. The purpose of this study was to determine the utility of applying a fragility analysis to randomized controlled trials (RCTs) evaluating distal radius fractures (DRFs).

      Methods

      We identified all dichotomous outcome data for randomized controlled trials of distal radius fractures (DRFs) published in 13 orthopedic journals from 2000 to 2020. The fragility index (FI) was determined by the number of event reversals required to change a P value from less than 0.05 to greater than 0.05, or vice-versa. The associated fragility quotient (FQ) was determined by dividing the FI by the sample size. The interquartile ranges (IQR) for the FI and FQ were calculated as the difference between the 25th and 75th percentiles for the overall study.

      Results

      Of the 7352 studies screened, 221 met the search criteria, with 34 randomized controlled trials evaluating distal radius fractures and 151 total outcome events included for analysis. The overall FI was 9 (IQR 6.5–11) with an associated fragility quotient of 0.097 (IQR 0.060–0.140). However, a majority of outcomes (78.8%) were not significant. Fragility analysis of statistically significant outcomes had an FI of 4 (IQR 1–10). All of the studies reported lost to follow-up (LTF) data where 20.6% reported lost to follow-up of greater or equal to 9.

      Conclusion

      The RCTs in distal radius fracture outcomes have comparable statistical robustness to literature in other orthopedic surgical specialties, are not as statistically stable as previously thought and should be interpreted with caution. We recommend that orthopedic literature report the FI and FQ along with the P values to help the reader draw reliable conclusions based on the fragility of outcome measures.

      Level of Evidence

      Level I

      Keywords

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Injury
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Porrino Jr, J.A.
        • Maloney E.
        • Scherer K.
        • et al.
        Fracture of the distal radius: epidemiology and premanagement 10 HAND 00(0) radiographic characterization.
        AJR Am J Roentgenol. 2014; 203: 551-559
        • Chen Y.
        • Chen X.
        • Li Z.
        • Yan H.
        • Zhou F.
        • Gao W.
        Safety and efficacy of operative versus nonsurgical management of distal radius fractures in elderly patients: a systematic review and meta-analysis.
        J Hand Surgery. 2016 Mar 1; 41: 404-413
        • Trumble T.E.
        • Schmitt S.R.
        • Vedder N.B.
        Factors affecting functional outcome of displaced intra-articular distal radius fractures.
        J Hand Surg Am. 1994; 19: 325-340
        • Ochen Y.
        • Peek J.
        • van der Velde D.
        • Beeres F.J.P.
        • van Heijl M.
        • Groenwold R.H.H.
        • Houwert R.M.
        • Heng M.
        Operative vs nonoperative treatment of distal radius fractures in adults: a systematic review and meta-analysis.
        JAMA Netw Open. 2020; 3e203497
        • Young B.T.
        • Rayan G.M.
        Outcome following nonoperative treatment of displaced distal radius fractures in low-demand patients older than 60 years.
        J Hand Surg Am. 2000; 25: 19-28
        • Pace V.
        • Lanzetti R.M.
        • Venditto T.
        • Park C.
        • Kim W.J.
        • Rinonapoli G.
        • Caraffa A.
        Dorsally displaced distal radius fractures: introduction of Pacetti's line as radiological measurement to predict dorsal fracture displacement.
        Acta Biomed. 2021 Jul 1; 92 (e2021200PMID: 34212906; PMCID)PMC8343761https://doi.org/10.23750/abm.v92i3.11392
      1. American Academy of Orthopaedic Surgeons. Management of distal radius fractures. Evidence-based clinical practice guideline. www.aaos.org/drfcpg. Published December 5, 2020.

      2. American Academy of Orthopaedic Surgeons. The treatment of distal radius fractures: guideline and evidence report. https://www.aaos.org/globalassets/quality-and-practiceresources/distal-radius/distal-radius-fractures-clinical-practice-guideline.pdf. Published December 5, 2009.

        • Jamnik A.A.
        • Chacon J.
        • Xiao A.X.
        • Wagner E.R.
        • Gottschalk M.B.
        The effect immobilization mechanisms have on radiographic outcomes and complication rates in the conservative treatment of distal radius fractures: a systematic review.
        Hand (N Y). 2022; (Apr 2Epub ahead of print. PMID: 35373590)15589447221081864https://doi.org/10.1177/15589447221081864
        • Zhang Y.X.
        • Li C.
        • Wang S.W.
        • Zhang M.L.
        • Zhang H.W.
        Volar plate fixation vs. non-operative management for distal radius fractures in older adults: a meta-analysis.
        Eur Rev Med Pharmacol Sci. 2021; 25 (JunPMID: 34156674): 3955-3966https://doi.org/10.26355/eurrev_202106_26036
        • Iyas A.M.
        • Jupiter J.B.
        Distal radius fractures–classification of treatment and indications for surgery.
        Hand Clin. 2010; 26: 37-42
        • Hohmann E.
        • Meta M.
        • Navalgund V.
        • Tetsworth K.
        The relationship between radiological alignment of united distal radius fractures and functional and patient-perceived outcomes in elderly patients.
        J Orthop Surg (Hong Kong). 2017; 252309499016684976
        • Cochrane A.L.
        Effectiveness and efficiency: random reflections on health services.
        London: Nuffield Provincial Hospitals Trust. 1972;
        • Eddy D.M
        Clinical decision making: from theory to practice. Connecting value and costs. Whom do we ask, and what do we ask them?.
        JAMA. 1990; 264 (Oct 3): 1737-1739
        • Djulbegovic B.
        • Guyatt G.H.
        Progress in evidence-based medicine: a quarter century on.
        Lancet. 2017; 390: 415-423
        • Wasserstein R.L.
        • Lazar N.A.
        The ASA statement on p-values: context, process and purpose.
        Am Stat. 2016; 70: 129-133
        • Parisien R.L.
        • Constant M.
        • Saltzman B.M.
        • Popkin C.A.
        • Ahmad C.S.
        • Li X.
        • et al.
        The fragility of statistical significance in cartilage restoration of the knee: a systematic review of randomized controlled trials.
        Cartilage. 2021; 13 (Dec1_suppl147S-55S)
        • Feinstein A.R.
        The unit fragility index: an additional appraisal of “statistical significance” for a contrast of two proportions.
        J Clin Epidemiol. 1990; 43: 201-209
        • Walsh M.
        • Srinathan S.K.
        • McAuley D.F.
        • Mrkobrada M.
        • Levine O.
        • Ribic C.
        • et al.
        The statistical significance of randomized controlled trials results is frequently fragile: a case for fragility index.
        J Clin Epidemiol. 2014; 67: 622-628
        • Ruzbarsky J.J.
        • Khormaee S.
        • Daluiski A.
        The fragility index in hand surgery randomized controlled trials.
        J Hand Surg Am. 2019 Aug 1; 44 (698-e1)
        • Evaniew N.
        • Files C.
        • Smith C.
        • Bhandari M.
        • Ghert M.
        • Walsh M.
        • et al.
        The fragility of statistically significant findings from randomized trials in spine surgery: a systematic survey.
        The Spine J. 2015; 15 (Oct 1): 2188-2197
        • Khormaee S.
        • Choe J.
        • Ruzbarsky J.J.
        • Agarwal K.N.
        • Blanco J.S.
        • Doyle S.M.
        • et al.
        The fragility of statistically significant results in pediatric orthopaedic randomized controlled trials as quantified by the fragility index: a systematic review.
        J Pediatric Orthopaedics. 2018; 38 (Sep 1): e418-e423
      3. Khan M., Evaniew N., Gichuru M., Habib A., Ayeni O.R., Bedi A., et al. The fragility of statistically significant findings from randomized trials in sports surgery: a systematic survey. The American journal of sports medicine. 2017 Jul;45(9):2164–70.

        • Tignanelli C.J.
        • Napolitano L.M.
        The fragility index in randomized clinical trials as a means of optimizing patient care.
        JAMA Surg. 2019; 154 (Jan 1): 74
        • Forrester L.A.
        • Jang E.
        • Lawson M.M.
        • Capi A.
        • Tyler W.K.
        Statistical fragility of surgical and procedural clinical trials in orthopaedic oncology.
        JAAOS Global Res Rev. 2020; 4 (Jun)
        • McCormick K.L.
        • Tedesco L.J.
        • Swindell H.W.
        • Forrester L.A.
        • Jobin C.M.
        • Levine W.N.
        Statistical fragility of randomized clinical trials in shoulder arthroplasty.
        J Shoulder Elbow Surg. 2021; 30 (Aug): 1787-1793
        • Parisien R.L.
        • Danford N.C.
        • Jarin I.J.
        • Li X.
        • Trofa D.P.
        • Vosseller J.T.
        The fragility of statistical findings in achilles tendon injury] research: a systematic review.
        J Am Acad Orthop Surg Glob Res Rev. 2021; 5 (Sep 2e21.00018)
        • Parisien R.L.
        • Trofa D.P.
        • O'Connor M.
        • Knapp B.
        • Curry E.J.
        • Tornetta III, P.
        • TS Lynch
        • X Li
        The fragility of significance in the hip arthroscopy literature: a systematic review.
        JBJS Open Access. 2021; 6 (Oct)
        • Parisien R.L.
        • Ehlers C.
        • Cusano A.
        • Tornetta P.
        • Li X.
        • Wang D
        The statistical fragility of platelet-rich plasma in rotator cuff surgery: a systematic review and meta-analysis.
        Am J Sports Med. 2021; (3rd) (Mar 1)363546521989976
        • Ruzbarsky J.J.
        • Khormaee S.
        • Rauck R.C.
        • Warren R.F.
        Fragility of randomized clinical trials of treatment of clavicular fractures.
        J Shoulder Elbow Surg. 2019; 28 (MarPMID: 30771826): 415-422https://doi.org/10.1016/j.jse.2018.11.039
        • Checketts J.X.
        • Scott J.T.
        • Meyer C.
        • Horn J.
        • Jones J.
        • Vassar M.
        The robustness of trials that guide evidence-based orthopedic surgery.
        J Bone Jt Surg Am. 2018; 100: e85