I am a big fan of the Rasch model and have already dedicated one post to Testing Rasch model in R with eRm. Today I would like to add one more post, because as much as I like R, for modelling latent variables I actually prefer MPlus. Hence, I want to show you how to test the Rasch model in MPlus.
For demonstrative purposes we will use the same data-set as in my previous Rasch model post. If you want to follow along on your own, then you can download the data and all input and output files here.
2-pl Model
We start with testing the less restrictive 2-pl model. In contrast to the Rasch model aka 1-pl model item discrimination parameter may vary in the 2-pl model.
We create the MPlus input file 00_2pl.inp and request the creation of an additional difftest file for later model comparison:
DATA:
file = rasch.dat;
VARIABLE:
names =
item1-item10
group
;
categorical =
item1-item10
;
usevar =
item1-item10
;
ANALYSIS:
estimator = WLSMV;
parameterization = theta;
MODEL:
!freely estimate all
!discrimination parameters
f1 by
item1* item2-item10
;
f1@1;
SAVEDATA:
difftest = 00_2pl_diff.dat;
Running 00_2pl.inp with MPlus gives us the output (abr.):
[...]
MODEL FIT INFORMATION
Number of Free Parameters 20
Chi-Square Test of Model Fit
Value 79.967*
Degrees of Freedom 35
P-Value 0.0000
* The chi-square value for MLM, MLMV, MLR, ULSMV, WLSM and WLSMV cannot be used
for chi-square difference testing in the regular way. MLM, MLR and WLSM
chi-square difference testing is described on the Mplus website. MLMV, WLSMV,
and ULSMV difference testing is done using the DIFFTEST option.
RMSEA (Root Mean Square Error Of Approximation)
Estimate 0.046
90 Percent C.I. 0.033 0.060
Probability RMSEA <= .05 0.658
CFI/TLI
CFI 0.977
TLI 0.971
Chi-Square Test of Model Fit for the Baseline Model
Value 2041.670
Degrees of Freedom 45
P-Value 0.0000
[...]
IRT PARAMETERIZATION IN TWO-PARAMETER PROBIT METRIC
WHERE THE PROBIT IS DISCRIMINATION*(THETA - DIFFICULTY)
Item Discriminations
F1 BY
ITEM1 0.562 0.097 5.765 0.000
ITEM2 1.039 0.130 8.016 0.000
ITEM3 1.145 0.124 9.256 0.000
ITEM4 0.973 0.107 9.101 0.000
ITEM5 0.900 0.101 8.918 0.000
ITEM6 1.303 0.155 8.381 0.000
ITEM7 0.941 0.105 8.998 0.000
ITEM8 1.010 0.114 8.847 0.000
ITEM9 0.921 0.124 7.410 0.000
ITEM10 0.907 0.107 8.497 0.000
Item Difficulties
ITEM1$1 -3.743 0.616 -6.074 0.000
ITEM2$1 -1.935 0.182 -10.617 0.000
ITEM3$1 -0.996 0.096 -10.321 0.000
ITEM4$1 -1.507 0.141 -10.700 0.000
ITEM5$1 -0.327 0.080 -4.073 0.000
ITEM6$1 -0.249 0.066 -3.767 0.000
ITEM7$1 0.110 0.075 1.464 0.143
ITEM8$1 0.473 0.079 5.974 0.000
ITEM9$1 1.287 0.135 9.536 0.000
ITEM10$1 2.294 0.248 9.253 0.000
Variances
F1 1.000 0.000 0.000 1.000The 2-pl model fits the data reasonably well. Further, except for item 1 all item discrimination parameters are very similar, which somewhat points to an 1-pl model.
1-pl Model aka Rasch Model
Initial Model
To convert the 2-pl to the 1-pl model, we fix all discrimination parameters to be equal. We save this change in 01_1pl.inp. Further, we request a χ² difference test, to compare both models.
DATA:
file = rasch.dat;
VARIABLE:
names =
item1-item10
group
;
categorical =
item1-item10
;
usevar =
item1-item10
;
ANALYSIS:
estimator = WLSMV;
parameterization = theta;
difftest = 00_2pl_diff.dat;
MODEL:
!fix all discrimination parameters
f1 by
item1* item2-item10 (fix)
;
f1@1;
OUTPUT:
mod;
SAVEDATA:
difftest = 01_1pl_diff.dat;Running 01_1pl.inp with MPlus gives us the output (abr.):
[...]
MODEL FIT INFORMATION
Number of Free Parameters 11
Chi-Square Test of Model Fit
Value 100.372*
Degrees of Freedom 44
P-Value 0.0000
Chi-Square Test for Difference Testing
Value 23.670
Degrees of Freedom 9
P-Value 0.0049
* The chi-square value for MLM, MLMV, MLR, ULSMV, WLSM and WLSMV cannot be used
for chi-square difference testing in the regular way. MLM, MLR and WLSM
chi-square difference testing is described on the Mplus website. MLMV, WLSMV,
and ULSMV difference testing is done using the DIFFTEST option.
RMSEA (Root Mean Square Error Of Approximation)
Estimate 0.046
90 Percent C.I. 0.034 0.058
Probability RMSEA <= .05 0.683
CFI/TLI
CFI 0.972
TLI 0.971
Chi-Square Test of Model Fit for the Baseline Model
Value 2041.670
Degrees of Freedom 45
P-Value 0.0000
[...]
IRT PARAMETERIZATION IN TWO-PARAMETER PROBIT METRIC
WHERE THE PROBIT IS DISCRIMINATION*(THETA - DIFFICULTY)
Item Discriminations
F1 BY
ITEM1 0.979 0.042 23.201 0.000
ITEM2 0.979 0.042 23.201 0.000
ITEM3 0.979 0.042 23.201 0.000
ITEM4 0.979 0.042 23.201 0.000
ITEM5 0.979 0.042 23.201 0.000
ITEM6 0.979 0.042 23.201 0.000
ITEM7 0.979 0.042 23.201 0.000
ITEM8 0.979 0.042 23.201 0.000
ITEM9 0.979 0.042 23.201 0.000
ITEM10 0.979 0.042 23.201 0.000
Item Difficulties
ITEM1$1 -2.622 0.163 -16.107 0.000
ITEM2$1 -1.993 0.126 -15.756 0.000
ITEM3$1 -1.072 0.088 -12.123 0.000
ITEM4$1 -1.503 0.102 -14.681 0.000
ITEM5$1 -0.313 0.074 -4.214 0.000
ITEM6$1 -0.283 0.074 -3.813 0.000
ITEM7$1 0.108 0.073 1.466 0.143
ITEM8$1 0.481 0.077 6.285 0.000
ITEM9$1 1.246 0.093 13.463 0.000
ITEM10$1 2.203 0.146 15.051 0.000
Variances
F1 1.000 0.000 0.000 1.000
[...]
MODEL MODIFICATION INDICES
NOTE: Modification indices for direct effects of observed dependent variables
regressed on covariates and residual covariances among observed dependent
variables may not be included. To include these, request MODINDICES (ALL).
Minimum M.I. value for printing the modification index 10.000
M.I. E.P.C. Std E.P.C. StdYX E.P.C.
BY Statements
F1 BY ITEM1 21.563 -0.619 -0.619 -0.442
F1 BY ITEM6 10.330 0.273 0.273 0.195
WITH Statements
ITEM10 WITH ITEM1 34.327 -0.635 -0.635 -0.635
ITEM10 WITH ITEM2 11.516 -0.285 -0.285 -0.285
Variances/Residual Variances
ITEM1 21.560 1.323 1.323 0.676
ITEM6 10.329 -0.626 -0.626 -0.320
[...]The model still fits reasonably well and now all item discrimination parameters are equal. Looking at the χ² difference test, however, we see that the restrictions yielded a significant (p < .01) worse fit. The modification indices confirm what was already hinted in the results of the 2-pl model – item 1 does not fit into the general pattern.
Adjusting the Model
To test whether assuming an equal discrimination parameter for item 1 is at fault for the significant χ² difference test, we remove the restriction for item 1 and run the model again.
In file 02_1pl_partial.inp we change the model to:
[...]
MODEL:
!fix all discrimination parameters,
!but for item1
f1 by
item1*
item2-item10 (fix)
;
!fix factor variance
!for identification
f1@1;
[...]This improves the fit and the χ² difference test is no longer significant:
[...]
MODEL FIT INFORMATION
Number of Free Parameters 12
Chi-Square Test of Model Fit
Value 80.388*
Degrees of Freedom 43
P-Value 0.0005
Chi-Square Test for Difference Testing
Value 8.956
Degrees of Freedom 8
P-Value 0.3460
* The chi-square value for MLM, MLMV, MLR, ULSMV, WLSM and WLSMV cannot be used
for chi-square difference testing in the regular way. MLM, MLR and WLSM
chi-square difference testing is described on the Mplus website. MLMV, WLSMV,
and ULSMV difference testing is done using the DIFFTEST option.
RMSEA (Root Mean Square Error Of Approximation)
Estimate 0.038
90 Percent C.I. 0.025 0.051
Probability RMSEA <= .05 0.937
CFI/TLI
CFI 0.981
TLI 0.980
Chi-Square Test of Model Fit for the Baseline Model
Value 2041.670
Degrees of Freedom 45
P-Value 0.0000
[...]Obviously, if we want the Rasch model to fit we have to remove item 1. We do so in 03_1pl_9items.inp:
DATA:
file = rasch.dat;
VARIABLE:
names =
item1-item10
group
;
categorical =
item2-item10
;
usevar =
item2-item10
;
ANALYSIS:
estimator = WLSMV;
parameterization = theta;
MODEL:
!fix all discrimination parameters
f1 by
item2* item3-item10 (fix)
;
!fix factor variance
!for identification
f1@1;
OUTPUT:
mod;Which nicely fits the data as can be seen in the output (abr.):
[...]
MODEL FIT INFORMATION
Number of Free Parameters 10
Chi-Square Test of Model Fit
Value 59.596*
Degrees of Freedom 35
P-Value 0.0059
* The chi-square value for MLM, MLMV, MLR, ULSMV, WLSM and WLSMV cannot be used
for chi-square difference testing in the regular way. MLM, MLR and WLSM
chi-square difference testing is described on the Mplus website. MLMV, WLSMV,
and ULSMV difference testing is done using the DIFFTEST option.
RMSEA (Root Mean Square Error Of Approximation)
Estimate 0.034
90 Percent C.I. 0.018 0.049
Probability RMSEA <= .05 0.964
CFI/TLI
CFI 0.987
TLI 0.987
Chi-Square Test of Model Fit for the Baseline Model
Value 1990.785
Degrees of Freedom 36
P-Value 0.0000
[...]
IRT PARAMETERIZATION IN TWO-PARAMETER PROBIT METRIC
WHERE THE PROBIT IS DISCRIMINATION*(THETA - DIFFICULTY)
Item Discriminations
F1 BY
ITEM2 1.007 0.044 23.080 0.000
ITEM3 1.007 0.044 23.080 0.000
ITEM4 1.007 0.044 23.080 0.000
ITEM5 1.007 0.044 23.080 0.000
ITEM6 1.007 0.044 23.080 0.000
ITEM7 1.007 0.044 23.080 0.000
ITEM8 1.007 0.044 23.080 0.000
ITEM9 1.007 0.044 23.080 0.000
ITEM10 1.007 0.044 23.080 0.000
Item Difficulties
ITEM2$1 -1.964 0.125 -15.775 0.000
ITEM3$1 -1.057 0.087 -12.150 0.000
ITEM4$1 -1.481 0.100 -14.760 0.000
ITEM5$1 -0.309 0.073 -4.215 0.000
ITEM6$1 -0.278 0.073 -3.811 0.000
ITEM7$1 0.106 0.072 1.466 0.143
ITEM8$1 0.474 0.075 6.294 0.000
ITEM9$1 1.228 0.091 13.519 0.000
ITEM10$1 2.171 0.143 15.188 0.000
Variances
F1 1.000 0.000 0.000 1.000
[...]Testing the Model across Multiple Groups
Now that the models fits the data in general we move on to the most important feature of the Rasch model – invariant comparisons. I.e., the relative positions of the item difficulty parameters are not allowed to differ across groups. In our data-set with have one grouping variable (group).
At first, let’s try whether the model still fits when the data is split into groups. In the file 04_1pl_9items_groups.inp we request a multigroup analysis by defining the grouping variable. MPlus’ default for multigroup analysis is strong measurement invariance, i.e. factor loadings and location parameters for CFA or analogously discrimination and difficulty parameters for IRT are held equal across groups. Hence, to allow the item difficulty parameters to differ between groups we have to explicitly request their estimation for the second group.
DATA:
file = rasch.dat;
VARIABLE:
names =
item1-item10
group
;
categorical =
item2-item10
;
usevar =
item2-item10
;
grouping =
group(1 = group1 2 = group2)
ANALYSIS:
estimator = WLSMV;
parameterization = theta;
MODEL:
!fix all discrimination parameters
f1 by
item2* item3-item10 (fix)
;
!fix factor variance
!for identification
f1@1;
!fix factor mean
!for identification
[f1@0];
!freely estimate difficulties
[item2$1-item10$1*];
model group2:
!freely estimate difficulties
[item2$1-item10$1*];
!fix residual variance
!to 1 (like in group1)
item2-item10@1;
OUTPUT:
mod;
SAVEDATA:
difftest = 04_1pl_9items_groups_diff.dat;
This yields (abr.):
[...]
MODEL FIT INFORMATION
Number of Free Parameters 19
Chi-Square Test of Model Fit
Value 106.507*
Degrees of Freedom 71
P-Value 0.0041
Chi-Square Contributions From Each Group
GROUP1 63.554
GROUP2 42.953
* The chi-square value for MLM, MLMV, MLR, ULSMV, WLSM and WLSMV cannot be used
for chi-square difference testing in the regular way. MLM, MLR and WLSM
chi-square difference testing is described on the Mplus website. MLMV, WLSMV,
and ULSMV difference testing is done using the DIFFTEST option.
RMSEA (Root Mean Square Error Of Approximation)
Estimate 0.041
90 Percent C.I. 0.023 0.056
Probability RMSEA <= .05 0.826
CFI/TLI
CFI 0.978
TLI 0.977
Chi-Square Test of Model Fit for the Baseline Model
Value 1658.288
Degrees of Freedom 72
P-Value 0.0000
[...]
IRT PARAMETERIZATION IN TWO-PARAMETER PROBIT METRIC
WHERE THE PROBIT IS DISCRIMINATION*(THETA - DIFFICULTY)
Group GROUP1
Item Discriminations
F1 BY
ITEM2 0.960 0.043 22.366 0.000
ITEM3 0.960 0.043 22.366 0.000
ITEM4 0.960 0.043 22.366 0.000
ITEM5 0.960 0.043 22.366 0.000
ITEM6 0.960 0.043 22.366 0.000
ITEM7 0.960 0.043 22.366 0.000
ITEM8 0.960 0.043 22.366 0.000
ITEM9 0.960 0.043 22.366 0.000
ITEM10 0.960 0.043 22.366 0.000
Means
F1 0.000 0.000 0.000 1.000
Item Difficulties
ITEM2$1 -1.673 0.147 -11.413 0.000
ITEM3$1 -0.871 0.115 -7.551 0.000
ITEM4$1 -1.322 0.129 -10.229 0.000
ITEM5$1 0.012 0.104 0.115 0.908
ITEM6$1 -0.024 0.104 -0.231 0.818
ITEM7$1 0.267 0.106 2.528 0.011
ITEM8$1 0.771 0.113 6.824 0.000
ITEM9$1 1.582 0.140 11.319 0.000
ITEM10$1 2.648 0.231 11.444 0.000
Variances
F1 1.000 0.000 0.000 1.000
Group GROUP2
Item Discriminations
F1 BY
ITEM2 0.960 0.043 22.366 0.000
ITEM3 0.960 0.043 22.366 0.000
ITEM4 0.960 0.043 22.366 0.000
ITEM5 0.960 0.043 22.366 0.000
ITEM6 0.960 0.043 22.366 0.000
ITEM7 0.960 0.043 22.366 0.000
ITEM8 0.960 0.043 22.366 0.000
ITEM9 0.960 0.043 22.366 0.000
ITEM10 0.960 0.043 22.366 0.000
Means
F1 0.000 0.000 0.000 1.000
Item Difficulties
ITEM2$1 -2.529 0.214 -11.804 0.000
ITEM3$1 -1.322 0.130 -10.132 0.000
ITEM4$1 -1.746 0.150 -11.677 0.000
ITEM5$1 -0.662 0.110 -6.003 0.000
ITEM6$1 -0.556 0.109 -5.116 0.000
ITEM7$1 -0.048 0.105 -0.461 0.644
ITEM8$1 0.218 0.105 2.070 0.038
ITEM9$1 0.989 0.119 8.344 0.000
ITEM10$1 1.936 0.167 11.608 0.000
Variances
F1 1.000 0.000 0.000 1.000
[...]
The model still fits the data well. In contrast to the earlier analyses we now see how much each group contributes to the χ² and we get separated parameter estimates for the groups.
Next, we fix the item difficulty parameters across groups to test whether they are equal and invariant comparisons hold true. In the file 05_1pl_9items_groups_final.inp we revoke the free estimation of the item difficulty parameters by commenting it out and instead allow the factor mean to be freely estimated in group 2. Fixing the difficulty parameters without freeing the mean would not only assume the same measurement model in both groups, but that the mean score in both groups is equal too.
model group2:
!fix difficulties
![item2$1-item10$1*];
!but free factor mean
[f1*];
!fix residual variance
!to 1 (like in group1)
item2-item10@1;
As a result we get (abr.):
[...]
THE MODEL ESTIMATION TERMINATED NORMALLY
MODEL FIT INFORMATION
Number of Free Parameters 11
Chi-Square Test of Model Fit
Value 114.107*
Degrees of Freedom 79
P-Value 0.0060
Chi-Square Contributions From Each Group
GROUP1 67.243
GROUP2 46.864
Chi-Square Test for Difference Testing
Value 8.435
Degrees of Freedom 8
P-Value 0.3922
* The chi-square value for MLM, MLMV, MLR, ULSMV, WLSM and WLSMV cannot be used
for chi-square difference testing in the regular way. MLM, MLR and WLSM
chi-square difference testing is described on the Mplus website. MLMV, WLSMV,
and ULSMV difference testing is done using the DIFFTEST option.
RMSEA (Root Mean Square Error Of Approximation)
Estimate 0.038
90 Percent C.I. 0.021 0.053
Probability RMSEA <= .05 0.893
CFI/TLI
CFI 0.978
TLI 0.980
Chi-Square Test of Model Fit for the Baseline Model
Value 1658.288
Degrees of Freedom 72
P-Value 0.0000
[...]
IRT PARAMETERIZATION IN TWO-PARAMETER PROBIT METRIC
WHERE THE PROBIT IS DISCRIMINATION*(THETA - DIFFICULTY)
Group GROUP1
Item Discriminations
F1 BY
ITEM2 0.960 0.043 22.366 0.000
ITEM3 0.960 0.043 22.366 0.000
ITEM4 0.960 0.043 22.366 0.000
ITEM5 0.960 0.043 22.366 0.000
ITEM6 0.960 0.043 22.366 0.000
ITEM7 0.960 0.043 22.366 0.000
ITEM8 0.960 0.043 22.366 0.000
ITEM9 0.960 0.043 22.366 0.000
ITEM10 0.960 0.043 22.366 0.000
Means
F1 0.000 0.000 0.000 1.000
Item Difficulties
ITEM2$1 -1.779 0.135 -13.215 0.000
ITEM3$1 -0.831 0.098 -8.501 0.000
ITEM4$1 -1.272 0.109 -11.641 0.000
ITEM5$1 -0.053 0.088 -0.601 0.548
ITEM6$1 -0.021 0.088 -0.240 0.811
ITEM7$1 0.379 0.089 4.273 0.000
ITEM8$1 0.763 0.093 8.184 0.000
ITEM9$1 1.551 0.112 13.849 0.000
ITEM10$1 2.535 0.166 15.311 0.000
Variances
F1 1.000 0.000 0.000 1.000
Group GROUP2
Item Discriminations
F1 BY
ITEM2 0.960 0.043 22.366 0.000
ITEM3 0.960 0.043 22.366 0.000
ITEM4 0.960 0.043 22.366 0.000
ITEM5 0.960 0.043 22.366 0.000
ITEM6 0.960 0.043 22.366 0.000
ITEM7 0.960 0.043 22.366 0.000
ITEM8 0.960 0.043 22.366 0.000
ITEM9 0.960 0.043 22.366 0.000
ITEM10 0.960 0.043 22.366 0.000
Means
F1 0.000 0.000 0.000 1.000
Item Difficulties
ITEM2$1 -2.317 0.148 -15.660 0.000
ITEM3$1 -1.369 0.108 -12.714 0.000
ITEM4$1 -1.810 0.121 -14.953 0.000
ITEM5$1 -0.592 0.092 -6.420 0.000
ITEM6$1 -0.560 0.091 -6.126 0.000
ITEM7$1 -0.159 0.089 -1.792 0.073
ITEM8$1 0.225 0.090 2.491 0.013
ITEM9$1 1.013 0.102 9.945 0.000
ITEM10$1 1.997 0.152 13.168 0.000
Variances
F1 1.000 0.000 0.000 1.000
[...]
The χ² difference test is not significant, so we assume that the item difficulty parameters are equal across groups. The displayed parameters however still differ. This is due to the fact that MPlus centers the means per group to zero for the IRT parametrization. This is in contrast to the output for the standard CFA parametrization (not shown here, but you can look it up in the output file), where you can see that all item thresholds which correspond to the item difficulty parameters are equal. Further, if you look closely at the item difficulty parameters, then you might notice that not only the ranks are the same across groups, but also the differences between items are exactly the same.
We can conclude that the item difficulty parameters are equivalent, but that the mean of the measured latent construct probably differs between groups. If we want to test this assumption, then we could compare the last model to an altered model with fixed mean in both groups. However, this is a story for another post.
Closing Remarks
I hope the given example was illustrative and can help you to carry out your own Rasch model analyses with MPlus.
If you have any questions or comments please post them in the comments section.
The address to download the data and all input and output files for rasch model is not available and can you give our readers a new one? Thank you!
ReplyDeleteI have fixed the link - thx for the hint
DeleteExcellent outline - I've been looking for something like this all day!
ReplyDelete