Skip to main content

Advertisement

Table 5 Quality assessment according to AHFMR 2004 (N = 8)

From: Reasons for and against participation in studies of medicinal therapies for women with breast cancer: a debate

Criteria Assessment* Remarks
Ellis et al 2002 [11] (Score = 0.71**)
Research question(s)/Objective(s) Yes At the beginning of the introduction; contains dependent/independent variables, including population
Study design Partially Not explicitly described, no inconsistencies in the subsequent data collection
Sampling Yes Described, exclusion criteria mentioned; written consent obtained in each case
Sample characteristics Yes Basic information given and depicted in tables
Randomization Partially Randomization performed; exact process of randomization not described
Blinding: Researchers n.a. Not possible
Blinding: Participants n.a. Not possible
Data collection Partially Categories of questionnaires given
Sample size Partially Power analysis performed; no information about low response rate of second questionnaire
Data analysis Partially Regression appropriate; no indication regarding the characteristics of participating/non-participating women
Variance estimate Yes Confidence intervals and ranges indicated
Confounding factor control Partially Control at analysis level with multivariate model, but no attempt to standardize the physician's consultation
Result depiction Partially Described in the text; the secondary result (change of knowledge) was not depicted graphically/in a table.
Conclusions Yes Relevant results are discussed and compared with other studies
Houlihan et al 2010 [12] (Score = 0.86**)
Research question(s)/Objective(s) Yes Listed
Study design Yes Stated
Sampling Partially Inclusion criteria mentioned, procedure not described
Sample characteristics Partially Information only regarding city and ethnic group given; average age of the women only in Discussion section (but without standard deviation); other basic data are not available
Randomization n.a. Not possible
Blinding: Researchers n.a. Not possible
Blinding: Participants n.a. Not possible
Data collection Partially Reproducible on a limited basis
Sample size Yes No power or variance analysis given; no problems with multiple tests described and significant values were obtained
Data analysis Yes Described
Variance estimate Yes Confidence intervals indicated
Confounding factor control Yes Control at analysis level with multivariate model
Result depiction Yes Described; significant results depicted in tables
Conclusions Yes Results depicted in a summarized manner and discussed with previous studies; limitations and recall bias reflected
Kim et al 2004 [13] (Score = 0.75**)
Research question(s)/Objective(s) Yes Clearly formulated and discussed throughout the text
Study design Partially That this is a non-comparative study only becomes obvious in the results section
Sampling Partially Described; random- but convenience sample
Sample characteristics Partially Basic data exist, but not tested regarding differences between the sub-groups
Randomization Partially Would have been possible
Blinding: Researchers n.a. Not possible
Blinding: Participants n.a. Not possible
Data collection Yes Reproducible
Sample size Yes Variance analysis; significant values obtained
Data analysis Partially Scenarios of conflicts of interest are listed; the questions/answer options/data analysis procedure described, results in the text do not match the table data
Variance estimate Yes Variance estimate performed
Confounding factor control Yes Control at analysis level with multivariate model
Result depiction Yes Results summarized and described in tables
Conclusions Partially Results discussed and compared with other studies; limits listed, possible influence on the results through the chance of winning 3 × $500 was not reflected
Lemieux et al 2008 [14] (Score = 0.82**)
Research question(s)/Objective(s) Yes At the beginning of the method section; contains variables to examine, population, place and timeframe
Study design Partially Not explicitly mentioned, but no inconsistencies result in the subsequent data collection
Sampling Partially Procedure described, exclusion criteria listed; but bias possible as the selection of cooperatives and pharmaceutical companies was made by experts
Sample characteristics Yes Basic information given, steps in the categorization mentioned
Randomization n.a. Not possible as the authors wanted to include all studies in Ontario from the years 1999-2002
Blinding: Researchers n.a. Not possible
Blinding: Participants n.a. Not possible
Data collection Yes Reproducible
Sample size Partially No power analysis
Data analysis Yes Poisson Regression appropriate; handling of missing values described
Variance estimate Yes Confidence intervals and ranges stated
Confounding factor control Partially Control at analysis level with multivariate model, but the institutions received money as incentive to participate in the study
Result depiction Yes Results depicted in the text and tables
Conclusions Yes Results discussed and compared with other studies
Loehberg et al 2010 [15] (Score = 0.77**)
Research question(s)/Objective(s) Partially At the end of the introduction; calculation of influencing factors through multiple regression analyses unclear
Study design Partially Not explicitly mentioned, no inconsistencies
Sampling Yes Procedure described, inclusion criteria mentioned
Sample characteristics Yes Basic information given
Randomization n.a. Not possible
Blinding: Researchers n.a. Not possible
Blinding: Participants n.a. Not possible
Data collection Partially Reproducible
Sample size Yes No power- or variance analysis given; no problems mentioned with multiple tests
Data analysis Yes Described
Variance estimate Yes Confidence intervals and distribution indicated
Confounding factor control Yes Analysis of sub-groups conducted
Result depiction Partially Results depicted in the text and tables do not all follow the objective of the article, but seem to be appropriate in general.
Conclusions Partially Results discussed and compared with other studies; no critical reflection that the information could possibly influence the number of participants
Mancini et al 2007 [16] (Score = 1.00**)
Research question(s)/Objective(s) Yes Described precisely
Study design Yes Described and appropriate
Sampling Yes Procedure described; inclusion criteria mentioned; consent forms collected
Sample characteristics Yes Basic information on the women and sub-groups given;
Randomization n.a. Not possible
Blinding: Researchers n.a. Not possible
Blinding: Participants n.a. Not possible
Data collection Yes Reproducible
Sample size Yes No power- or variance analysis, sample size seems to be sufficiently large
Data analysis Yes Logistical regression analysis appropriate, individual tests within the framework of descriptive statistics conducted; significance level mentioned
Variance estimate Yes Confidence intervals and standard deviations given
Confounding factor control Yes Control at analysis level with multivariate model
Result depiction Yes Results of regression analysis are listed in the text and table; not all significant results of the comparisons within the framework of descriptive statistics were also described in the text, but no inconsistencies result
Conclusions Yes Results discussed and compared with other studies
Mandelblatt et al 2005 [17] (Score = 0.79**)
Research question(s)/Objective(s) Partially Described in the abstract, formulation of objective in the text fails to mention the investigation of two interventions
Study design Yes Described and appropriate
Sampling Yes Procedure described; inclusion criteria mentioned; consent forms collected
Sample characteristics Yes Basic information on the women and sub-groups given
Randomization Yes Randomization performed; procedure described
Blinding: Researchers n.a. Not possible
Blinding: Participants n.a. Not possible
Data collection Yes Reproducible
Sample size Yes Not obvious whether a power analysis was conducted later; the sample size seems to be sufficient
Data analysis Partially Logistical regression analysis appropriate, tests conducted within the framework of descriptive statistic analysis are not mentioned; handling of missing values described; significance level not given
Variance estimate Yes Confidence intervals, standard deviations and range given
Confounding factor control Partially Control at analysis level with multivariate model; but in asking the control group, the same standardized questionnaire as in the intervention group was used
Result depiction Partially Secondary results depicted in the text and tables; but the primary result is not mentioned in the text, it is only listed in the table; recording of influencing factors not conducted for both interventions. Differences regarding the objective breast cancer risk between those women who consented to participation in the medication study and those who refused participation were not pointed out. The text only lists percentages and no absolute numbers; consequently, readers have to infer from the tables how many women in total participated in the data collection, and subsequently in the medication study.
Conclusions Partially Results discussed but only partially compared with other studies
Randonina et al 2008 [18] (Score = 0.91**)
Research question(s)/Objective(s) Yes Appears in the middle of the method section, primary and secondary results are mentioned at the end of the introduction
Study design Partially Not mentioned explicitly for this collection process, only the design of the medication study is described
Sampling Yes Procedure described; including criteria mentioned; consent forms collected
Sample characteristics Yes Basic information on the women and sub-groups given
Randomization n.a. Not possible
Blinding: Researchers n.a. Not possible
Blinding: Participants n.a. Not possible
Data collection Yes Reproducible
Sample size Yes Power analysis given
Data analysis Yes Analysis appropriate and apparent from the objective; handling of missing values described; significance level given
Variance estimate Yes Confidence intervals, standard deviations and ranges given
Confounding factor control Yes Control at analysis level with multivariate model; also, control regarding age distribution based on low response rate for questionnaires; furthermore, an attempt was made to standardize the preceding consultation session
Result depiction Partially Results depicted in the text and tables and correspond with each other; but results of the regression analysis are interpreted in the text as "in connection with" and not as "influence" - the interpretation in the framework of the remark in the table is again depicted correctly
Conclusions Yes Results discussed and compared with other studies
  1. * Possible categories: yes, partially, no, not applicable
  2. ** Ascertainment according to AHFMR 2004 (p. 14) = ((number "yes" * 2) + (number "partially"" * 1))/(28 - (number "n.a." *2))