Clearly a lot of effort has gone into this Paper. Its empirical findings can be supported by theory.
Wright & Masters (1982) "Rating Scale Analysis" page 100 is the definitive algebraic and statistical statement on polytomous Rasch mean-square fit statistics and their t-tests. That page is reproduced (with permission) at RMT 3:4.
The theoretical relationship between mnsq, t, and sample size is shown in the Figure in RMT 17:1 - which pictures the Wilson-Hilferty transformation mentioned in the Paper's text.
The Paper's Figures support the contention that, for most polytomous Rasch applications, the Infit and Outfit fit statistics provide the same guidance, so only Outfit (the more conventional mean-square) needs to be published - which could have been one of the Paper's conclusions.
The Paper's conclusions are
"the t-statistics are sample size dependent,"
Yes, the Figure in RMT 17:1 indicates that the t-test is usefully sensitive in the sample-size range of 30 to 300 cases.
and
"sample size invariance appears to exist for the mean square fit statistics." -
this is only true if unmodeled noise is relatively homogeneous across samples of different sizes from the same population (which is not a claim of the Rasch model). But that situation is common enough for this conclusion to be a useful rule of thumb. It is not true when increasing the sample size requires bringing in cases from less controlled situations, e.g., an international study which starts in Europe, then includes Asia, then Africa.
Competing interests
Have also published in this area and developed some of the software used by the authors.
More about the theory behind this Paper
16 June 2008
Clearly a lot of effort has gone into this Paper. Its empirical findings can be supported by theory.
Wright & Masters (1982) "Rating Scale Analysis" page 100 is the definitive algebraic and statistical statement on polytomous Rasch mean-square fit statistics and their t-tests. That page is reproduced (with permission) at RMT 3:4.
The theoretical relationship between mnsq, t, and sample size is shown in the Figure in RMT 17:1 - which pictures the Wilson-Hilferty transformation mentioned in the Paper's text.
The Paper's Figures support the contention that, for most polytomous Rasch applications, the Infit and Outfit fit statistics provide the same guidance, so only Outfit (the more conventional mean-square) needs to be published - which could have been one of the Paper's conclusions.
The Paper's conclusions are
"the t-statistics are sample size dependent,"
Yes, the Figure in RMT 17:1 indicates that the t-test is usefully sensitive in the sample-size range of 30 to 300 cases.
and
"sample size invariance appears to exist for the mean square fit statistics." -
this is only true if unmodeled noise is relatively homogeneous across samples of different sizes from the same population (which is not a claim of the Rasch model). But that situation is common enough for this conclusion to be a useful rule of thumb. It is not true when increasing the sample size requires bringing in cases from less controlled situations, e.g., an international study which starts in Europe, then includes Asia, then Africa.
Competing interests
Have also published in this area and developed some of the software used by the authors.