rel_irt {lazy.irt}

R Documentation

Calculation of Test Reliability and SEM under IRT

Description

Calculation of Test Reliability and SEM under IRT

Usage

rel_irt(
  param,
  weight = NULL,
  maxscore = 100,
  npoints = 51,
  thmin = -4,
  thmax = 4,
  thdist = 1,
  print = 1,
  plot = 1
)

Arguments

param	Item Parameter Data Frame
weight	Weight data frame
maxscore	The maximum score to be used to calculate var(X) etc.
npoints	# of discrete points for theta
thmin	Minimum value of discrete thata value
thmax	Maximum value of discrete thata value
thdist	Type of theta distribution = 0 to use uniform, = 1 to use N(0,1)
print	= 1 to print result
plot	= 1 or 2 to plot result

Details

Test reliability of a text Z is defined as:

 rel(Z) = var(T) / var(Z) = 1 - var(E)/var(Z)

where T and E are, resp, the true score and the error score of Z.
For observed score: X = trf(theta) + E and E(E|theta) = 0,

var(E) = E(var(E|theta)) = E(var(X|theta))
var(X) = E(var(X|theta)) + var(E(X|theta)) = var(E) + var(trf)
var(T) = var(trf)

For thetahat based score: Y = thetahat = theta + error and E(error|theta)=0,

var(error) = E(var(error|theta)) = var(Y|theta) = 1/info(theta)
var(Y) = E(var(Y|theta)) + var(E(Y|theta)) = E(1/info(theta)) + var(theta)
var(theta) = 1 or var(uniform)

The standard error of measurement, SEM, is defined as,
For X

 SEMx(X) = sqrt( var(E|X) )  or  SEMx(theta) = sqrt( var(E|trf(theta)) )

For Y,

 SEMt(Y) = sqrt( var(error|theta) ) or
 SEMt(X) = sqrt( var(error|trf(theta)) )

Their averages are denoted, resp, as
aSEM or aSEM_theta.
Note that, SEMx(X) and SEMx(theta) are convex and SEMt(Y) and SEMt(theta) are concave.

Note that the value depends on the range and the shape of the theta distribution.
Try changing npoints, thmin and thmax.

Value

A list of:

thmin, thmax, npoints, thdist
SigmaX2 Observed Score Variance
SigmaT2 True Score Variance
SigmaT22 Shoulld be the same as above
SigmaE2 Error Score Variance
aSEM Average Standard Error of Measurement
rel The reliability coefficient

SigmaX_theta2 Variance of Thetahat
SigmaT_theta2 Variance of Theta
SigmaE_theta2 Average Variance of Thetahat
aSEM_theta Average Standard Error of Measurement of Thetahat
rel_theta The reliability coefficient of Thetahat based Score

SigmaX_xs2 Variance of X in [0,maxscore]
SigmaT_xs2 Variance of T in [0,maxscore]
SigmaE_xs2 Average Variance of E in [0,maxscore]
aSEM_xs Average Standard Error of Measurement of X in [0,maxscore]
rel_xs The reliability coefficient

SigmaX_ys2 Variance of Thetahat in [0,maxscore]
SigmaT_ys2 Variance of Theta in [0,maxscore]
SigmaE_ys2 Average Variance of Thetahat given theta in [0,maxscore]
aSEM_ys Average Standard Error of Measurement of Thetahat in [0,maxscore]
rel_ys The reliability coefficient

SEM_x SEM of X at True Score
SEM_t SEM of X and Y(thetahat) at Theta
SEM_x100 Rescaled SEM of X and Y(thetahat) at Rescaled True Score
SEM_t100 Rescaled SEM of X and Y(thetahat) at theta

References

Lord, F. M. (1980). Applications of Item Response Theory To Practical Testing Problems. Lawrence Erlbaum Associates. Mahwah, New Jergey

Samejima, F. (1994) Estimation of Reliability Coefficients Using the Test Information Function and Its Modifications. Applied Psychological Measurement. 18, 229-244.

Toyoda. H. (1989) The Methods for Estimating the Reliability Coefficient under Item Response Model. (in Japanese) Japanese Journal of Educational Psychology. vol. 37, 283-285.

Examples

res <- rel_irt( paramA1, thmin=-4, thmax=4, plot=2 )
res <- rel_irt( paramA1, thmin=-4, thmax=4, thdist=0, plot=2 )

---