Table 5 Quality assessment according to AHFMR 2004 (N = 8)

Ellis et al 2002 [11] (Score = 0.71**)

Research question(s)/Objective(s)

Yes

At the beginning of the introduction; contains dependent/independent variables, including population

Study design

Partially

Not explicitly described, no inconsistencies in the subsequent data collection

Sampling

Yes

Described, exclusion criteria mentioned; written consent obtained in each case

Sample characteristics

Yes

Basic information given and depicted in tables

Randomization

Partially

Randomization performed; exact process of randomization not described

Blinding: Researchers

n.a.

Not possible

Blinding: Participants

n.a.

Not possible

Data collection

Partially

Categories of questionnaires given

Sample size

Partially

Power analysis performed; no information about low response rate of second questionnaire

Data analysis

Partially

Regression appropriate; no indication regarding the characteristics of participating/non-participating women

Variance estimate

Yes

Confidence intervals and ranges indicated

Confounding factor control

Partially

Control at analysis level with multivariate model, but no attempt to standardize the physician's consultation

Result depiction

Partially

Described in the text; the secondary result (change of knowledge) was not depicted graphically/in a table.

Conclusions

Yes

Relevant results are discussed and compared with other studies

Houlihan et al 2010 [12] (Score = 0.86**)

Research question(s)/Objective(s)

Yes

Listed

Study design

Yes

Stated

Sampling

Partially

Inclusion criteria mentioned, procedure not described

Sample characteristics

Partially

Information only regarding city and ethnic group given; average age of the women only in Discussion section (but without standard deviation); other basic data are not available

Randomization

n.a.

Not possible

Blinding: Researchers

n.a.

Not possible

Blinding: Participants

n.a.

Not possible

Data collection

Partially

Reproducible on a limited basis

Sample size

Yes

No power or variance analysis given; no problems with multiple tests described and significant values were obtained

Data analysis

Yes

Described

Variance estimate

Yes

Confidence intervals indicated

Confounding factor control

Yes

Control at analysis level with multivariate model

Result depiction

Yes

Described; significant results depicted in tables

Conclusions

Yes

Results depicted in a summarized manner and discussed with previous studies; limitations and recall bias reflected

Kim et al 2004 [13] (Score = 0.75**)

Research question(s)/Objective(s)

Yes

Clearly formulated and discussed throughout the text

Study design

Partially

That this is a non-comparative study only becomes obvious in the results section

Sampling

Partially

Described; random- but convenience sample

Sample characteristics

Partially

Basic data exist, but not tested regarding differences between the sub-groups

Randomization

Partially

Would have been possible

Blinding: Researchers

n.a.

Not possible

Blinding: Participants

n.a.

Not possible

Data collection

Yes

Reproducible

Sample size

Yes

Variance analysis; significant values obtained

Data analysis

Partially

Scenarios of conflicts of interest are listed; the questions/answer options/data analysis procedure described, results in the text do not match the table data

Variance estimate

Yes

Variance estimate performed

Confounding factor control

Yes

Control at analysis level with multivariate model

Result depiction

Yes

Results summarized and described in tables

Conclusions

Partially

Results discussed and compared with other studies; limits listed, possible influence on the results through the chance of winning 3 × $500 was not reflected

Lemieux et al 2008 [14] (Score = 0.82**)

Research question(s)/Objective(s)

Yes

At the beginning of the method section; contains variables to examine, population, place and timeframe

Study design

Partially

Not explicitly mentioned, but no inconsistencies result in the subsequent data collection

Sampling

Partially

Procedure described, exclusion criteria listed; but bias possible as the selection of cooperatives and pharmaceutical companies was made by experts

Sample characteristics

Yes

Basic information given, steps in the categorization mentioned

Randomization

n.a.

Not possible as the authors wanted to include all studies in Ontario from the years 1999-2002

Blinding: Researchers

n.a.

Not possible

Blinding: Participants

n.a.

Not possible

Data collection

Yes

Reproducible

Sample size

Partially

No power analysis

Data analysis

Yes

Poisson Regression appropriate; handling of missing values described

Variance estimate

Yes

Confidence intervals and ranges stated

Confounding factor control

Partially

Control at analysis level with multivariate model, but the institutions received money as incentive to participate in the study

Result depiction

Yes

Results depicted in the text and tables

Conclusions

Yes

Results discussed and compared with other studies

Loehberg et al 2010 [15] (Score = 0.77**)

Research question(s)/Objective(s)

Partially

At the end of the introduction; calculation of influencing factors through multiple regression analyses unclear

Study design

Partially

Not explicitly mentioned, no inconsistencies

Sampling

Yes

Procedure described, inclusion criteria mentioned

Sample characteristics

Yes

Basic information given

Randomization

n.a.

Not possible

Blinding: Researchers

n.a.

Not possible

Blinding: Participants

n.a.

Not possible

Data collection

Partially

Reproducible

Sample size

Yes

No power- or variance analysis given; no problems mentioned with multiple tests

Data analysis

Yes

Described

Variance estimate

Yes

Confidence intervals and distribution indicated

Confounding factor control

Yes

Analysis of sub-groups conducted

Result depiction

Partially

Results depicted in the text and tables do not all follow the objective of the article, but seem to be appropriate in general.

Conclusions

Partially

Results discussed and compared with other studies; no critical reflection that the information could possibly influence the number of participants

Mancini et al 2007 [16] (Score = 1.00**)

Research question(s)/Objective(s)

Yes

Described precisely

Study design

Yes

Described and appropriate

Sampling

Yes

Procedure described; inclusion criteria mentioned; consent forms collected

Sample characteristics

Yes

Basic information on the women and sub-groups given;

Randomization

n.a.

Not possible

Blinding: Researchers

n.a.

Not possible

Blinding: Participants

n.a.

Not possible

Data collection

Yes

Reproducible

Sample size

Yes

No power- or variance analysis, sample size seems to be sufficiently large

Data analysis

Yes

Logistical regression analysis appropriate, individual tests within the framework of descriptive statistics conducted; significance level mentioned

Variance estimate

Yes

Confidence intervals and standard deviations given

Confounding factor control

Yes

Control at analysis level with multivariate model

Result depiction

Yes

Results of regression analysis are listed in the text and table; not all significant results of the comparisons within the framework of descriptive statistics were also described in the text, but no inconsistencies result

Conclusions

Yes

Results discussed and compared with other studies

Mandelblatt et al 2005 [17] (Score = 0.79**)

Research question(s)/Objective(s)

Partially

Described in the abstract, formulation of objective in the text fails to mention the investigation of two interventions

Study design

Yes

Described and appropriate

Sampling

Yes

Procedure described; inclusion criteria mentioned; consent forms collected

Sample characteristics

Yes

Basic information on the women and sub-groups given

Randomization

Yes

Randomization performed; procedure described

Blinding: Researchers

n.a.

Not possible

Blinding: Participants

n.a.

Not possible

Data collection

Yes

Reproducible

Sample size

Yes

Not obvious whether a power analysis was conducted later; the sample size seems to be sufficient

Data analysis

Partially

Logistical regression analysis appropriate, tests conducted within the framework of descriptive statistic analysis are not mentioned; handling of missing values described; significance level not given

Variance estimate

Yes

Confidence intervals, standard deviations and range given

Confounding factor control

Partially

Control at analysis level with multivariate model; but in asking the control group, the same standardized questionnaire as in the intervention group was used

Result depiction

Partially

Secondary results depicted in the text and tables; but the primary result is not mentioned in the text, it is only listed in the table; recording of influencing factors not conducted for both interventions. Differences regarding the objective breast cancer risk between those women who consented to participation in the medication study and those who refused participation were not pointed out. The text only lists percentages and no absolute numbers; consequently, readers have to infer from the tables how many women in total participated in the data collection, and subsequently in the medication study.

Conclusions

Partially

Results discussed but only partially compared with other studies

Randonina et al 2008 [18] (Score = 0.91**)

Research question(s)/Objective(s)

Yes

Appears in the middle of the method section, primary and secondary results are mentioned at the end of the introduction

Study design

Partially

Not mentioned explicitly for this collection process, only the design of the medication study is described

Sampling

Yes

Procedure described; including criteria mentioned; consent forms collected

Sample characteristics

Yes

Basic information on the women and sub-groups given

Randomization

n.a.

Not possible

Blinding: Researchers

n.a.

Not possible

Blinding: Participants

n.a.

Not possible

Data collection

Yes

Reproducible

Sample size

Yes

Power analysis given

Data analysis

Yes

Analysis appropriate and apparent from the objective; handling of missing values described; significance level given

Variance estimate

Yes

Confidence intervals, standard deviations and ranges given

Confounding factor control

Yes

Control at analysis level with multivariate model; also, control regarding age distribution based on low response rate for questionnaires; furthermore, an attempt was made to standardize the preceding consultation session

Result depiction

Partially

Results depicted in the text and tables and correspond with each other; but results of the regression analysis are interpreted in the text as "in connection with" and not as "influence" - the interpretation in the framework of the remark in the table is again depicted correctly

Conclusions

Yes

Results discussed and compared with other studies