Skip to main content

Table 2 Challenges facing researchers conducting an IMPF

From: Individual participant data meta-analysis of prognostic factor studies: state of the art?

Identifying all relevant studies

 ·

 Unavailability of IPD in some studies

 ·

 Time-consuming and costly nature of obtaining, cleaning and analysing the IPD.

Issues within individual studies

 ·

 Dealing with skewed continuous variables and possible outliers.

 ·

 Inability of IPD to overcome deficiencies of original studies, such as being retrospective rather than prospective, being too small for a multivariable analysis, missing important confounders, missing participant data or being of low methodological quality, etc.

 ·

 How to assess the quality of studies identified

 ·

 Re-analysing individual study IPD before considering meta-analysis. For a summary of important issues for the analysis of single prognostic factor studies see Holländer and Sauerbrei [9]. The re-analysis of a single study as the preliminary or first step toward a meta-analysis is influenced by and has consequences for the meta-analysis strategy (15).

Heterogeneity between studies

 ·

 Different definitions of disease or outcome; e.g. Noordzij et al.[44] note different definitions of hypocalcemia across studies, whilst the MeRGE [40] collaborators note the definition of acute myocardial infarction changed over time.

 ·

 Different participant inclusion and exclusion criteria

 ·

 Different methods of measuring the same prognostic factor, for example see difficulties described by Look et al [2].

 ·

 For survival data different lengths of follow-up

 ·

 Factors measured at different points in time or at different stages of disease across studies; e.g. the MeRGE [40] collaborators note that the timing of echocardiography was variable in their included studies, although within 2 weeks of the index acute myocardial infarction

 ·

 Different (or out-dated) treatments strategies, especially when a mixture of older and newer studies are combined; e.g. Yap et al. [36] state that a large proportion of the patients in their included trials did not receive common post-myocardial infarction therapy such as β-blockers and ACE inhibitor.

 ·

 Insufficent information about treatment for some of the studies.

Statistical issues for meta-analysis

 ·

 Missing data, including: missing factor values and outcome data for some participants within a study, and unavailable factors in some studies

 ·

 Inability to adjust prognostic effects for a consistent set of adjustment factors in each study

 ·

 Different measurement techniques between studies may be acceptable for adjustment variables, but are critical for the variable of main interest

 ·

 Insufficient information to separate patient outcomes more discretely, e.g. Thakkinstian et al. [37] could not separate chronic allograft nephropathy from graft rejection or acute rejection from chronic rejection

 ·

 Imposed choice of cut-off levels when individual studies categorise their continuous variables and/or categorise their continuous outcomes in their provided IPD

 ·

 Difficulty in using a continuous scale for continuous factors in meta-analysis when some studies give IPD values on a continuous cale and others do not (e.g. see Rovers et al. [43])

 ·

 Considering whether it is sensible and/or possible to investigate differential prognostic effects in subgroups

 ·

 Potential for study-level confounding when assessing whether study covariates (e.g. year of publication) modify the prognostic effect.

 ·

 Difficulty of interpreting summary meta-analysis results in the presence of heterogeneity (and heterogeneous populations) across studies.

Assessment of potential biases

 ·

 Potential for publication bias and availability bias

 ·

 How to assess the robustness of IPD meta-analysis results to the inclusion/exclusion of studies only providing summary data; and how to combine IPD studies with summary data studies