Skip to main content

Table 3 Proposal for ways to assess aspects of measurement properties of GAS in a blinded placebo-controlled RCT

From: Goal attainment scaling as an outcome measure in rare disease trials: a conceptual proposal for validation

COSMIN Domain

Quality aspect

COSMIN definition

Definition of quality aspect as applicable to GAS

Measurable on GAS step

How to assess on individual level***

How to assess on trial level

Validity

Content validity

The degree to which the content of an HR-PRO instrument is an adequate reflection of the construct to be measured

The degree to which goals capture patient outcomes that can be expected to be influenced by treatment.

Selecting goals [1]

Evaluation of the goals by independent clinical experts****

 

Content validity

The degree to which the content of an HR-PRO instrument is an adequate reflection of the construct to be measured

Levels are determined consistently, ensuring that they are centered around an appropriate value of zero and proportionally ordered.

Defining the levels [2]

Evaluation of the goal achievement levels by independent clinical experts****

 

Construct validity

The degree to which the scores of an HR-PRO instrument are consistent with hypotheses (for instance with regard to internal relationships, relationships to scores of other instruments, or differences between relevant groups) based on the assumption that the HRPRO instrument validly measures the construct to be measured

The underlying construct (one or more than one for every patient) is the attainment of goals that are chosen to reflect the effect that is expected from an intervention on a functional level. Construct validity is the degree to which the scores of GAS are consistent with hypotheses.

Assessing goal attainment [3]

Hypothesis testing: Comparison of scores on individual goals with change scores on other measurement instruments that measure a construct that is similar to one or more of the chosen goals, such as a function scale or a balance test*

Hypothesis testing: Comparison between two groups. It is expected that the mean GAS scores of two randomized groups receiving effective or non-effective interventions will differ in favor of the group receiving the effective intervention.

Reliability

Intra rater reliability

The extent to which scores for patients who have not changed are the same for repeated measurement by the same persons on different occasions

Goal attainment is assessed consistently when performed by the same rater for the same patient in the same condition repeatedly (possibly hypothetically).

Assessing goal attainment [3]

Repeated assessment of video recording of the assessment of the goal attainment level ** by the same rater

 

Inter rater reliability

The extent to which scores for patients who have not changed are the same for repeated measurement by different persons on the same occasions

Goal attainment is assessed consistently when performed by different raters for the same patient in the same condition.

Assessing goal attainment [3]

Assessment of the video recording of the assessment of the goal attainment level** by one or more independent raters

 

Inter trial reliability

 

Goal attainment scaling leads to consistent results when implemented in repeated implementations of the same trial.

Assessing goal attainment [3]

 

Replication between trials or within one trial (with a split-half design); comparison of mean differences between intervention groups

  1. * It is not feasible to compare all chosen goals with other measurement instruments, as goals may vary widely in a heterogeneous population
  2. ** In the video recording the goal score should not be made explicit
  3. *** The score of GAS is only a change score, and cannot be interpreted causally at the individual level. However, content validity of GAS can only be assessed on an individual level, since every patient individually chooses his or her own goals. The same goes for reliability and aspects of our proposed construct validity: since all goals differ on an individual level, this has to be assessed at the individual level, in order to ensure the validity and interpretability of GAS on group level
  4. **** According to the original COSMIN methodology, this should be done very extensively with qualitative interviews and/or focus groups. However, in the setting of rare diseases, this is not feasible