Advertisement

Natural experiments for orthopaedic trauma research: An introduction

Open AccessPublished:November 10, 2022DOI:https://doi.org/10.1016/j.injury.2022.11.028

      Abstract

      Natural experiments are observational studies of medical treatments in which treatment allocation is determined by factors outside the control of the investigators, arguably resembling experimental randomisation. Natural experiments in the field of orthopaedic trauma research are scarce. However, they have great potential due to the process governing treatment allocation and the existence of opposing treatment strategies between hospitals or between regions as a result of local education, conviction, or cultural and socio-economic factors. Here, the possibilities and opportunities of natural experiments in the orthopaedic trauma field are discussed. Potential solutions are presented to improve the validity of natural experiments and how to assess the credibility of such studies. Above all, it is meant to spark a discussion about its role within the field of orthopaedic trauma research.

      Keywords

      Introduction

      Randomised clinical trials (RCT) are widely accepted as the highest level of evidence for evaluating and comparing effectiveness of therapeutic interventions [
      • Sacks H.
      • Chalmers T.C.
      • Smith Jr., H.
      Randomized versus historical controls for clinical trials.
      ]. Nevertheless, results of traditional RCTs may have limited generalizability; the field of orthopaedic trauma research is no exception to that. The potential limited generalizability stems from the highly artificial conditions that are usually imposed on surgical practice to fit the randomised study design [
      • Beks R.B.
      • Bhashyam A.R.
      • Houwert R.M.
      • van der Velde D.
      • van Heijl M.
      • Smeeing D.P.J.
      • et al.
      When observational studies are as helpful as randomized trials: examples from orthopedic trauma.
      ]. Surgeons frequently have a strong personal preference for a certain treatment due to technical skills, personal experience, and local culture and infrastructure [
      • Houwert R.M.
      • Beks R.B.
      • Dijkgraaf M.G.W.
      • Roes K.C.B.
      • Oner F.C.
      • Hietbrink F.
      • et al.
      Study methodology in trauma care: towards question-based study designs.
      ]. These aspects play an important role in the surgeons’ decision whether or not to include patients into an RCT [
      • Beks R.B.
      • Bhashyam A.R.
      • Houwert R.M.
      • van der Velde D.
      • van Heijl M.
      • Smeeing D.P.J.
      • et al.
      When observational studies are as helpful as randomized trials: examples from orthopedic trauma.
      ,
      • McCulloch P.
      • Taylor I.
      • Sasako M.
      • Lovett B.
      • Griffin D.
      Randomised trials in surgery: problems and possible solutions.
      ]. In addition, patients frequently have a strong opinion as well about treatment options, particularly when it comes down to fundamentally different treatments, such as non-operative care and surgical treatment. This also contributes to selective inclusion of participants in surgical trials. What is more, the time from presentation of concept at a congress to publication of RCTs in orthopaedic trauma research is on average 10 years, which is highly undesirable in a fast-developing field like orthopaedic trauma surgery [
      • Axelrod D.
      • Trask K.
      • Buckley R.E.
      • Johal H.
      The Canadian Orthopaedic Trauma Society.
      ].
      Observational studies are increasingly regarded to provide evidence that is complementary to that from RCTs, provided the observational studies are of sufficient quality [
      • Concato J.
      • Shah N.
      • Horwitz R.I.
      Randomized, controlled trials, observational studies, and the hierarchy of research designs.
      ,
      • Abraham N.S.
      • Byrne C.J.
      • Young J.M.
      • Solomon M.J.
      Meta-analysis of well-designed nonrandomized comparative studies of surgical procedures is as good as randomized controlled trials.
      ]. In contrast to RCTs, observational studies are often more representative of daily clinical practice. In addition, they are less costly and generate evidence much faster than traditional RCTs. However, due to the absence of randomisation, incomparability of treatment groups may occur leading to confounding bias. Natural experiments, a particular type of observational study, might provide a solution. In this paper, we describe different aspects of natural experiment studies, with a focus on natural experiments in orthopaedic trauma research. We discuss issues that need to be considered when conducting, reporting, or reading about natural experiment studies.

      Natural experiments

      In natural experiment studies, the potential for confounding might be substantially less than in other observational studies [
      • Craig P.
      • Katikireddi S.V.
      • Leyland A.
      • Popham F.
      Natural experiments: an overview of methods, approaches, and contributions to public health intervention research.
      ,
      • Uddin M.J.
      • Groenwold R.H.
      • Ali M.S.
      • de Boer A.
      • Roes K.C.
      • Chowdhury M.A.
      • et al.
      Methods to control for unmeasured confounding in pharmacoepidemiology: an overview.
      ]. Natural experiments are observational studies in which patients are exposed to either the experimental or the control condition, whereby treatment allocation is determined by factors outside the control of the investigators. The process governing treatment allocation arguably resembles the random assignment in an experimental setting, hence the name natural experiment.
      An example of a natural experiment could be a comparison of treatment strategies, where differences in strategies exist between hospitals or regions as a result of local education and conviction, cultural, and socio-economic differences. Generally, trauma patients will receive acute care from the nearest hospital able to facilitate adequate treatment, which is determined by the geographical location of the incident: “what you get, depends on where you live and who you see.”(10, 11) The exact location of their accident – and thus the hospital they are referred to – is, to a large extent, considered independent of the characteristics of those patients [
      • Fanuele J.
      • Koval K.J.
      • Lurie J.
      • Zhou W.
      • Tosteson A.
      • Ring D.
      Distal radial fracture treatment: what you get may depend on your age and address.
      ,
      • Koval K.J.
      • Lurie J.
      • Zhou W.
      • Sparks M.B.
      • Cantu R.V.
      • Sporer S.M.
      • et al.
      Ankle fractures in the elderly: what you get depends on where you live and who you see.
      ,
      • Stadhouder A.
      • Buskens E.
      • de Klerk L.W.
      • Verhaar J.A.
      • Dhert W.A.
      • Verbout A.J.
      • et al.
      Traumatic thoracic and lumbar spinal fractures: operative or nonoperative treatment: comparison of two treatment strategies by means of surgeon equipoise.
      ]. Hence, different trauma care facilities are expected to treat similar groups of patients. This is an ideal starting point to compare treatment strategies across hospitals or regions in a natural experiment setting [
      • Stadhouder A.
      • Buskens E.
      • de Klerk L.W.
      • Verhaar J.A.
      • Dhert W.A.
      • Verbout A.J.
      • et al.
      Traumatic thoracic and lumbar spinal fractures: operative or nonoperative treatment: comparison of two treatment strategies by means of surgeon equipoise.
      ]. For the remainder, we will refer to these hospitals or regions with opposing treatment regimens simply as “schools”.

      Examples of natural experiments in orthopaedic trauma

      An illustrative example of a natural experiment is the study by Hauschild et al. [
      • Hauschild O.
      • Konrad G.
      • Audige L.
      • de Boer P.
      • Lambert S.M.
      • Hertel R.
      • et al.
      Operative versus non-operative treatment for two-part surgical neck fractures of the proximal humerus.
      ] They compared non-operative with operative treatment for proximal humerus fractures by comparing patients across four hospitals in Switzerland. One of the participating centres consistently offered non-operative treatment to all their patients while the other three performed surgery on all their patients with proximal humerus fractures. Patients were very similar across the treatment groups, providing the possibility to make valid comparisons.
      Another example of a natural experiment is a study by Stadhouder et al. [
      • Stadhouder A.
      • Buskens E.
      • de Klerk L.W.
      • Verhaar J.A.
      • Dhert W.A.
      • Verbout A.J.
      • et al.
      Traumatic thoracic and lumbar spinal fractures: operative or nonoperative treatment: comparison of two treatment strategies by means of surgeon equipoise.
      ], who used information available in medical records to compare patients hospitalised for traumatic spinal fractures in two university trauma centres in the Netherlands. One of these centres had a long-established treatment strategy of non-operative care and performed surgery on rare occasions. In the other centre, patients more often received operative care in case of traumatic spinal fractures. Since the patient groups that were admitted to either of the two hospitals were very similar, this allowed for a comparison between treatment strategies.
      A third example is a natural experiment in the form of a pre-post design (or before-after design). Schoenfeld et al. compared 14 040 patients with femoral neck fractures prior to implementation of a new healthcare reform in Massachusetts with 9445 patients after implementation with regard to cost-effectiveness and complications [
      • Schoenfeld A.J.
      • Weaver M.J.
      • Power R.K.
      • Harris M.B.
      Does Health Reform Change Femoral Neck Fracture Care? A Natural Experiment in the State of Massachusetts.
      ]. Again, baseline characteristics between the two groups of patients were remarkably similar.

      Methodological challenges of a natural experiment

      Defining the research question

      When designing a natural experiment, the first step, as in all research, is to define a clear research question. It should be articulated which treatment regimens are compared (intervention as well as comparator), in what patient population with which clinical condition (study population) and clearly define the outcome of interest (primary and secondary outcomes). A frequently used structure to articulate a research question, is the so-called PICO (Table 1).
      Table 1PICO for orthopaedic trauma research.
      PICOMinimal set of items to report/asses (if applicable):
      Population
      • -
        Anatomical location fracture
      • -
        Type of fracture (open/closed, simple/multifragmentary or combination/all)
      • -
        Age group
      InterventionIn case of surgical treatment:
      • -
        Osteosynthesis material
      • -
        Surgical approach
      • -
        Postoperative treatment (type&duration)
      In case of conservative treatment:
      • -
        Type of conservative treatment (including duration)
      ComparatorIn case of surgical treatment:
      • -
        Osteosynthesis material
      • -
        Surgical approach
      • -
        Postoperative treatment (type&duration)
      In case of conservative treatment:
      • -
        Type of conservative treatment (including duration)
      Outcome- What is the outcome?

      - When is the outcome assessed?

      - How is the outcome assessed?
      In orthopaedic trauma research the intervention and comparator are often defined only by the nature of the treatment itself (non-operative or operative) and, in case of operative treatment, the surgical technique. It is important to also incorporate aftercare into these elements of the research questions for several reasons. Aftercare strategies may differ between hospitals or regions. Additionally, they are part of the treatment strategies patients receive and may impact clinical and functional outcomes. In the context of a natural experiment, they are part of the ‘school’ that patients are exposed to and thus should be clearly defined.
      Clearly defining clinical outcomes is also important. In orthopaedic trauma, many clinical outcomes can be measured objectively and are frequently based on events requiring (operative or medical) interventions, radiological, biochemical or microbiological outcome data. A clear outcome definition should include a time component (when is it measured) and manner in which it is measured, which is frequently neglected in current literature [
      • Paul Pynsent J.F.
      • Carr Andrew
      Outcome measures in orthopaedics and orthopaedic trauma.
      ].

      Design

      For natural experiments in orthopaedic trauma there are certain design elements that should be considered to maximise its potential. The backbone of this design is formed by a treatment allocation process that is (to a large extent) independent of patient characteristics [
      • Fanuele J.
      • Koval K.J.
      • Lurie J.
      • Zhou W.
      • Tosteson A.
      • Ring D.
      Distal radial fracture treatment: what you get may depend on your age and address.
      ,
      • Koval K.J.
      • Lurie J.
      • Zhou W.
      • Sparks M.B.
      • Cantu R.V.
      • Sporer S.M.
      • et al.
      Ankle fractures in the elderly: what you get depends on where you live and who you see.
      ]. The archetype of a natural experiment in orthopaedic trauma is a comparison between hospitals where different treatment protocols are implemented, while referrals to the different hospitals are independent of patient characteristics (i.e., similar patient “case-mix” across hospitals). Preferably these "schools" consistently provide one of the treatment options to all (or the majority) of their patients with the clinical condition of interest. When performing a natural experiment, it is important that researchers convincingly argue that treatment allocation is indeed independent of individual patient characteristics, rather than trying to find convincing arguments in the comparison of baseline characteristics between treatment groups.
      Important to note is that the setting of the ‘’schools’’ should be similar in order to prevent relevant case-mix differences (i.e., difference in characteristics of their treated patients) possibly leading to confounding. For natural experiments in orthopaedic trauma care this means that the ‘’schools’’ should provide the same level of trauma care (level I, II or III) and be located in regions with the same socio-economic development. Essentially, both ‘’schools’’ should be comparable to such a degree that it is plausible to assume that school A could have provided the treatment from school B, and vice versa, if their conviction on optimal treatment had been different. Once these ‘’schools’’ have been identified, they can be used to compare the treatments of interest.
      All eligible patients should be registered, including patients that are excluded in order to gain insight on possible selection mechanisms by comparing patient characteristics between included and excluded patients. Patients should be treated according to the local preference and conviction of the "schools" with regard to the optimal treatment for their clinical condition.
      A natural experiment study can be performed retrospectively as well as prospectively. The advantages of a prospective design are that follow-up and measurement of baseline characteristics and outcomes can be pre-specified and standardised across the "schools", thus reducing the potential for information bias. Fig. 1 illustrates the necessary steps. Patients with the clinical condition of interest are identified through a hospital records search (retrospective design) or during their visit at the emergency department/outpatient clinic (prospective design) in participating hospitals representing different "schools" of intervention (school A and B). Data on baseline characteristics and the clinical condition of each patient should be collected.
      Fig. 1
      Fig. 1Flowchart of patient inclusion for the natural experiment including the restriction method using clinical equipoise as criterion).

      Comparability of treatment groups

      Even though natural experiments aim to compare different “schools” across, for example, different hospitals, patient groups may differ between schools in more respects than only the treatment strategies under study. To the extent that such differences in potential confounding variables are measured, this can be controlled for in the analysis, such as in any observational study of treatment effects. It is therefore of the utmost importance to collect data on key prognostic patient characteristics, as these will be needed in the statistical analysis of the study to correct for possible confounding (Fig. 1). This advice holds irrespective of whether data are collected retrospectively or prospectively, be it that in prospective studies it may be possible to ensure that information about confounding variables is collected in a standardized manner and possibly the proportion of data being missing is smaller than in retrospective studies using routinely collected data for example based on electronic patient records [
      • Bots S.H.
      • Groenwold R.H.H.
      • Dekkers O.M.
      Using electronic health record data for clinical research: a quick guide.
      ].
      Conventional methods to correct for (measured) confounding include stratification, regression adjustment, and matching. Another approach to reduce the amount of confounding is to restrict the study population by using clinical equipoise as an inclusion criterion [
      • Stadhouder A.
      • Buskens E.
      • de Klerk L.W.
      • Verhaar J.A.
      • Dhert W.A.
      • Verbout A.J.
      • et al.
      Traumatic thoracic and lumbar spinal fractures: operative or nonoperative treatment: comparison of two treatment strategies by means of surgeon equipoise.
      ]. In practise, this could be achieved by presenting all relevant data of eligible patients to an independent expert panel blinded for actual treatment received. The expert panel should consist of representatives from both “schools” of intervention. The panel is asked to decide independently on the preferred treatment for the eligible patient as if patients were presented to them in clinical practice. Patients are included if there is disagreement on treatment choice between the "schools"; they are excluded in case of agreement (Fig. 1). This ensures that the included study population consists of patients that would have received treatment A in "school A” but were in fact seen and treated according to the conviction of "school B", and vice versa (exchangeability). This way, the patients for whom the panel agrees regarding preferred treatment strategy, which are generally patients with very distinct disease or patient characteristics driving treatment preference unanimously in one direction, will be excluded. By restricting the study population to those for whom there is clinical equipoise, the potential impact of confounding is reduced. Clinical equipoise as inclusion criterion can be used both in prospective and retrospective natural experiment studies.
      Regarding the use of an expert panel and clinical equipoise for inclusion of patients, there are different options to implement this in a study. For example, patients could be included if at least 20% of panel members disagree with the other 80% of panel members. (1:4 distribution amongst experts). This means that patients are eligible for inclusion if (in a panel consisting of, for example, five experts) 4 (or fewer) experts prefer treatment A, while 1 (or more) prefer treatment B for a given patient. Such a threshold could be based on a study that assessed at which proportion of agreement on the merit of a new treatment amongst ethical committee members, the members perceived the conduct of a trial investigating the new treatment as ethically responsible (the level of collective equipoise) [
      • Mhaskar R.
      • BB B.
      • Djulbegovic B.
      At what level of collective equipoise does a randomized clinical trial become ethical for the members of institutional review board/ethical committees?.
      ].
      Importantly, the use of clinical equipoise as inclusion criterion is expected to reduce the number of patients included in the study. In the aforementioned study by Stadhouder et al., 190 of the 636 patients (30%) could be included based on this criterion [
      • Stadhouder A.
      • Buskens E.
      • de Klerk L.W.
      • Verhaar J.A.
      • Dhert W.A.
      • Verbout A.J.
      • et al.
      Traumatic thoracic and lumbar spinal fractures: operative or nonoperative treatment: comparison of two treatment strategies by means of surgeon equipoise.
      ]. The addition of clinical equipoise as inclusion criteria should therefore not be seen as a necessity but rather an extension of the natural experiment design to further improve comparability of treatment groups.
      Irrespective of whether this restriction method is used or not, it is important to assess the distributions of baseline characteristics across the treatment groups. This provides insight into the apparent comparability of "schools" and whether clinical equipoise as inclusion criterion has proved successful in creating comparable treatment groups. Nevertheless, known confounders could still be accounted for, for example through a multivariable regression analysis or propensity score analysis [

      Hernán M.A. RJCIWIBRCHC, 2006.

      ].

      Reporting of natural experiments

      The STROBE statement is a checklist of items that should be reported on in papers about observational studies [
      • von Elm E.
      • Altman D.G.
      • Egger M.
      • Pocock S.J.
      • Gotzsche P.C.
      • Vandenbroucke J.P.
      • et al.
      The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting of observational studies.
      ]. The RECORD statement is a reporting guideline for studies using routinely collected data [
      • von Elm E.
      • Altman D.G.
      • Egger M.
      • Pocock S.J.
      • Gotzsche P.C.
      • Vandenbroucke J.P.
      • et al.
      The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting of observational studies.
      ]. Many of the items mentioned in these reporting guidelines are also applicable to natural experiments. Some items, however, require specific attention when reporting on natural experiments for orthopaedic trauma. In case of a comparison between ‘’schools’’, it is essential to give insight whether participating schools offer only one treatment (‘’pure school’’), or both treatment modalities under investigation, but with a distinct preference of one treatment over the other (‘’majority school’’). In the latter situation, proportions of applied treatments within schools should be reported on. What is more, arguments should be provided to support the assumption of comparability of patient groups across different schools. In addition, details about the compared strategies should be reported, including peri‑operative care and after-treatment, except perhaps in case these are according to (international) standards, in which case a reference to those standards would be sufficient.

      Discussion

      Natural experiments in the field of orthopaedic trauma are still uncommon [
      • Beks R.B.
      • Ochen Y.
      • Frima H.
      • Smeeing D.P.J.
      • van der Meijden O.
      • Timmers T.K.
      • et al.
      Operative versus nonoperative treatment of proximal humeral fractures: a systematic review, meta-analysis, and comparison of observational studies and randomized controlled trials.
      ,
      • Smeeing D.P.J.
      • van der Ven D.J.C.
      • Hietbrink F.
      • Timmers T.K.
      • van Heijl M.
      • Kruyt M.C.
      • et al.
      Surgical versus nonsurgical treatment for midshaft clavicle fractures in patients aged 16 years and older: a systematic review, meta-analysis, and comparison of randomized controlled trials and observational studies.
      ,
      • van de Wall B.J.M.
      • Ochen Y.
      • Beeres F.J.P.
      • Babst R.
      • Link B.C.
      • Heng M.
      • et al.
      Conservative vs. operative treatment for humeral shaft fractures: a meta-analysis and systematic review of randomized clinical trials and observational studies.
      ,
      • Ochen Y.
      • Beks R.B.
      • van Heijl M.
      • Hietbrink F.
      • Leenen L.P.H.
      • van der Velde D.
      • et al.
      Operative treatment versus nonoperative treatment of Achilles tendon ruptures: systematic review and meta-analysis.
      ]. Nevertheless, this study design has great potential in this field compared to traditional observational study designs. Under the conditions outlined above (specifically regarding comparability of treatment groups), evidence obtained through natural experiments may be complementary to the evidence obtained through randomised trials. In particular, in orthopaedic trauma, patients are exposed to high variability of surgical decision making caused by strong convictions by surgeons as “surgeons agree mostly with themselves, and not so much with each other” [
      • Doornberg J.N.
      • Rademakers M.V.
      • van den Bekerom M.P.
      • Kerkhoffs G.M.
      • Ahn J.
      • Steller E.P.
      • et al.
      Two-dimensional and three-dimensional computed tomography for the classification and characterisation of tibial plateau fractures.
      ]. In natural experiments, this variability is turned into an advantage, by using it as the basis of a comparison between treatment strategies [
      • Houwert R.M.
      • Beks R.B.
      • Dijkgraaf M.G.W.
      • Roes K.C.B.
      • Oner F.C.
      • Hietbrink F.
      • et al.
      Study methodology in trauma care: towards question-based study designs.
      ]. All pros and cons of natural experiments are described in Table 2.
      Table 2Pros and cons of natural experiments in orthopaedic trauma research.
      ProsCons
      InexpensiveDependant on natural variation for feasibility of study
      High patient accrual rateNo true randomization (does not rule out unmeasured confounding)
      Generalisability of results
      Good internal validity due to quasi-randomisation
      Conduct research that is otherwise unethical.
      According the Oxford level of evidence natural experiments are categorized as observational cohort studies, thus traditionally considered level 2b [
      • Durieux N.
      • Vandenput S.
      • Pasleau F.
      [OCEBM levels of evidence system].
      ]. It should, however, be acknowledged that natural experiments differ from traditional observational studies by the fact that confounding is addressed in both the study design (school comparison) and analysis stage (correction for confounders) in contrast to traditional observational studies that generally only perform the latter. By incorporating a measure to limit confounding in the design, they share more similarities with randomised clinical trials than traditional observational studies; hence also the similarity in nomenclature between “natural experiment” and the alternative name for randomised clinical trial, “randomised experiment”. The most pronounced differences between randomised clinical trials, natural experiments and traditional observational studies are described in Table 3.
      Table 3Differences between randomised clinical trials, traditional observational studies and natural experiments in orthopaedic trauma research.
      Randomised clinical trialsTraditional observational studiesNatural experiments
      Exposure (intervention)Intervention that may differ from clinical practice. Usually two interventions included in trial.Standard clinical practice.Standard clinical practice.
      PopulationOften restricted to younger and relatively healthy patients.Can include entire population.Can include entire population.
      Confounding controlControl for both measured and unmeasured confounding through randomisation.Control for measured confounding through statistical correction. No control for unmeasured confounding.Control for measured confounding through statistical correction. Control (to unknown extent) for unmeasured confounding through school comparison.
      CostsExpensiveOften inexpensiveOften inexpensive
      Time frameTime consuming due to partial inclusion of patients.Often fast as most patients are included.Often fast as most patients are included.
      OutcomeStandardised measurement of endpoints.Measurement of endpoints restricted by routine clinical practice.Standardised measurement of endpoints.
      Blinding patientPossibleNot possibleNot possible
      Blinding outcome assessorPossibleUnusualPossible
      We would like to stress the importance of a proper sample size calculation as integral part in conducting a natural experiment [
      • Nayak B.K.
      Understanding the relevance of sample size calculation.
      ]. One can only draw a precise and accurate conclusion with a sufficiently large sample size. A smaller sample will give a result which may not be sufficiently powered to detect a difference between the groups and the study may turn out to be falsely negative leading to a type II error. Natural experiments follow the same standard approach to sample size calculation as any other empirical study [
      • Craig P.
      • Katikireddi S.V.
      • Leyland A.
      • Popham F.
      Natural experiments: an overview of methods, approaches, and contributions to public health intervention research.
      ]. Also in natural experiments, the sample size calculation is based on the primary endpoint of interest.
      Both the annual incidence of the clinical condition of interest and estimated proportion that are expected to be included by using clinical equipoise as inclusion criterion, play a vital role in evaluating feasibility of the planned natural experiment. In order to estimate the proportion that may be included when using clinical equipoise as inclusion criterion, one can measure the amount of disagreement in the expert panel prior to conducting the study. Basically, this can be done by subjecting clinical data of, for example, 12 random historical patients with the clinical condition of interest to the expert panel from the opposing "schools". The amount of disagreement reflects the proportion of all patients with the clinical condition of interest that can be included in the study. As described previously, the addition of clinical equipoise as inclusion criteria should not be seen as a necessity but rather an extension of the natural experiment design if conditions allow the inclusion of this design-element into the study.
      In recent years several prospective natural experiments have been initiated by the Natural Experiments (NEXT) Study Group. The NEXT Study Group is an international non-profit collaboration of clinical researchers in the field of emergency and (orthopaedic) trauma surgery. The ambition of the NEXT Study Group is to contribute to the improvement of patient care by collecting relevant evidence through international natural experiments. Ongoing studies include the OPVENT study comparing non-operative care to surgical treatment for multiple rib fractures and the LADON proximal humerus study also comparing non-operative and operative treatment strategies. In the LADON study clinical equipoise is used as an additional inclusion criterion [
      • Beks R.B.
      • de Jong M.B.
      • Sweet A.
      • Peek J.
      • van Wageningen B.
      • Tromp T.
      • et al.
      Multicentre prospective cohort study of nonoperative versus operative treatment for flail chest and multiple rib fractures after blunt thoracic trauma: study protocol.
      ,

      Nonoperative versus Operative Treatment for proximal humeral fractures. An International multicentre prospective cohort study (LADON). Netherlands Trial Register Trial NL9357, 2022.

      ].
      Orthopaedic trauma is a fast-developing field requiring study designs that deliver high quality evidence and, most of all, can keep up with ongoing developments within the field. This manuscript discusses the possibilities of natural experiments as a means to provide valuable evidence and how to assess the credibility of such studies within the orthopaedic trauma field. Above all, it is meant to spark a discussion about its role within our research field.

      Declaration of Competing Interest

      None.

      References

        • Sacks H.
        • Chalmers T.C.
        • Smith Jr., H.
        Randomized versus historical controls for clinical trials.
        Am J Med. 1982; 72: 233-240
        • Beks R.B.
        • Bhashyam A.R.
        • Houwert R.M.
        • van der Velde D.
        • van Heijl M.
        • Smeeing D.P.J.
        • et al.
        When observational studies are as helpful as randomized trials: examples from orthopedic trauma.
        J Trauma Acute Care Surg. 2019; 87: 730-732
        • Houwert R.M.
        • Beks R.B.
        • Dijkgraaf M.G.W.
        • Roes K.C.B.
        • Oner F.C.
        • Hietbrink F.
        • et al.
        Study methodology in trauma care: towards question-based study designs.
        Eur J Trauma Emerg Surg. 2021; 47: 479-484
        • McCulloch P.
        • Taylor I.
        • Sasako M.
        • Lovett B.
        • Griffin D.
        Randomised trials in surgery: problems and possible solutions.
        BMJ. 2002; 324: 1448-1451
        • Axelrod D.
        • Trask K.
        • Buckley R.E.
        • Johal H.
        The Canadian Orthopaedic Trauma Society.
        Bone Joint J. 2021; 103-B: 898-901
        • Concato J.
        • Shah N.
        • Horwitz R.I.
        Randomized, controlled trials, observational studies, and the hierarchy of research designs.
        N Engl J Med. 2000; 342: 1887-1892
        • Abraham N.S.
        • Byrne C.J.
        • Young J.M.
        • Solomon M.J.
        Meta-analysis of well-designed nonrandomized comparative studies of surgical procedures is as good as randomized controlled trials.
        J Clin Epidemiol. 2010; 63: 238-245
        • Craig P.
        • Katikireddi S.V.
        • Leyland A.
        • Popham F.
        Natural experiments: an overview of methods, approaches, and contributions to public health intervention research.
        Annu Rev Public Health. 2017; 38: 39-56
        • Uddin M.J.
        • Groenwold R.H.
        • Ali M.S.
        • de Boer A.
        • Roes K.C.
        • Chowdhury M.A.
        • et al.
        Methods to control for unmeasured confounding in pharmacoepidemiology: an overview.
        Int J Clin Pharm. 2016; 38: 714-723
        • Fanuele J.
        • Koval K.J.
        • Lurie J.
        • Zhou W.
        • Tosteson A.
        • Ring D.
        Distal radial fracture treatment: what you get may depend on your age and address.
        J Bone Joint Surg Am. 2009; 91: 1313-1319
        • Koval K.J.
        • Lurie J.
        • Zhou W.
        • Sparks M.B.
        • Cantu R.V.
        • Sporer S.M.
        • et al.
        Ankle fractures in the elderly: what you get depends on where you live and who you see.
        J Orthop Trauma. 2005; 19: 635-639
        • Stadhouder A.
        • Buskens E.
        • de Klerk L.W.
        • Verhaar J.A.
        • Dhert W.A.
        • Verbout A.J.
        • et al.
        Traumatic thoracic and lumbar spinal fractures: operative or nonoperative treatment: comparison of two treatment strategies by means of surgeon equipoise.
        Spine (Phila Pa 1976). 2008; 33: 1006-1017
        • Hauschild O.
        • Konrad G.
        • Audige L.
        • de Boer P.
        • Lambert S.M.
        • Hertel R.
        • et al.
        Operative versus non-operative treatment for two-part surgical neck fractures of the proximal humerus.
        Arch Orthop Trauma Surg. 2013; 133: 1385-1393
        • Schoenfeld A.J.
        • Weaver M.J.
        • Power R.K.
        • Harris M.B.
        Does Health Reform Change Femoral Neck Fracture Care? A Natural Experiment in the State of Massachusetts.
        J Orthop Trauma. 2015; 29: 494-499
        • Paul Pynsent J.F.
        • Carr Andrew
        Outcome measures in orthopaedics and orthopaedic trauma.
        CRC Press, 2004
        • Bots S.H.
        • Groenwold R.H.H.
        • Dekkers O.M.
        Using electronic health record data for clinical research: a quick guide.
        Eur J Endocrinol. 2022;
        • Mhaskar R.
        • BB B.
        • Djulbegovic B.
        At what level of collective equipoise does a randomized clinical trial become ethical for the members of institutional review board/ethical committees?.
        Acta Inform Med. 2013; 21: 156-159
      1. Hernán M.A. RJCIWIBRCHC, 2006.

        • von Elm E.
        • Altman D.G.
        • Egger M.
        • Pocock S.J.
        • Gotzsche P.C.
        • Vandenbroucke J.P.
        • et al.
        The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting of observational studies.
        Internist. 2008; 49: 688-693
        • Beks R.B.
        • Ochen Y.
        • Frima H.
        • Smeeing D.P.J.
        • van der Meijden O.
        • Timmers T.K.
        • et al.
        Operative versus nonoperative treatment of proximal humeral fractures: a systematic review, meta-analysis, and comparison of observational studies and randomized controlled trials.
        J Shoulder Elbow Surg. 2018; 27: 1526-1534
        • Smeeing D.P.J.
        • van der Ven D.J.C.
        • Hietbrink F.
        • Timmers T.K.
        • van Heijl M.
        • Kruyt M.C.
        • et al.
        Surgical versus nonsurgical treatment for midshaft clavicle fractures in patients aged 16 years and older: a systematic review, meta-analysis, and comparison of randomized controlled trials and observational studies.
        Am J Sports Med. 2017; 45: 1937-1945
        • van de Wall B.J.M.
        • Ochen Y.
        • Beeres F.J.P.
        • Babst R.
        • Link B.C.
        • Heng M.
        • et al.
        Conservative vs. operative treatment for humeral shaft fractures: a meta-analysis and systematic review of randomized clinical trials and observational studies.
        J Shoulder Elbow Surg. 2020;
        • Ochen Y.
        • Beks R.B.
        • van Heijl M.
        • Hietbrink F.
        • Leenen L.P.H.
        • van der Velde D.
        • et al.
        Operative treatment versus nonoperative treatment of Achilles tendon ruptures: systematic review and meta-analysis.
        BMJ. 2019; 364: k5120
        • Doornberg J.N.
        • Rademakers M.V.
        • van den Bekerom M.P.
        • Kerkhoffs G.M.
        • Ahn J.
        • Steller E.P.
        • et al.
        Two-dimensional and three-dimensional computed tomography for the classification and characterisation of tibial plateau fractures.
        Injury. 2011; 42: 1416-1425
        • Durieux N.
        • Vandenput S.
        • Pasleau F.
        [OCEBM levels of evidence system].
        Rev Med Liege. 2013; 68: 644-649
        • Nayak B.K.
        Understanding the relevance of sample size calculation.
        Indian J Ophthalmol. 2010; 58: 469-470
        • Beks R.B.
        • de Jong M.B.
        • Sweet A.
        • Peek J.
        • van Wageningen B.
        • Tromp T.
        • et al.
        Multicentre prospective cohort study of nonoperative versus operative treatment for flail chest and multiple rib fractures after blunt thoracic trauma: study protocol.
        BMJ Open. 2019; 9e023660
      2. Nonoperative versus Operative Treatment for proximal humeral fractures. An International multicentre prospective cohort study (LADON). Netherlands Trial Register Trial NL9357, 2022.