Continuous norming of psychometric tests: A simulation study of parametric and semi-parametric approaches
Autoři:
Alexandra Lenhard aff001; Wolfgang Lenhard aff002; Sebastian Gary aff002
Působiště autorů:
Test Development Center, Psychometrica, Dettelbach, Bavaria, Germany
aff001; Institute of Psychology, University of Wuerzburg, Bavaria, Germany
aff002
Vyšlo v časopise:
PLoS ONE 14(9)
Kategorie:
Research Article
doi:
https://doi.org/10.1371/journal.pone.0222279
Souhrn
Continuous norming methods have seldom been subjected to scientific review. In this simulation study, we compared parametric with semi-parametric continuous norming methods in psychometric tests by constructing a fictitious population model within which a latent ability increases with age across seven age groups. We drew samples of different sizes (n = 50, 75, 100, 150, 250, 500 and 1,000 per age group) and simulated the results of an easy, medium, and difficult test scale based on Item Response Theory (IRT). We subjected the resulting data to different continuous norming methods and compared the data fit under the different test conditions with a representative cross-validation dataset of n = 10,000 per age group. The most significant differences were found in suboptimal (i.e., too easy or too difficult) test scales and in ability levels that were far from the population mean. We discuss the results with regard to the selection of the appropriate modeling techniques in psychometric test construction, the required sample sizes, and the requirement to report appropriate quantitative and qualitative test quality criteria for continuous norming methods in test manuals.
Klíčová slova:
Research and analysis methods – Simulation and modeling – Biology and life sciences – Psychology – Psychometrics – Social sciences – People and places – Population groupings – Age groups – Physical sciences – Mathematics – Probability theory – Probability distribution – Skewness – Normal distribution – Statistical distributions – Algebra – Polynomials – Statistics – Statistical models
Zdroje
1. American Psychiatric Association, American Psychiatric Association, Editor. Diagnostic and statistical manual of mental disorders: DSM-5. 5th ed. Washington, D.C: American Psychiatric Association; 2013.
2. Rutter M, Tizard J, Yule W, Graham P, Whitmore K. Isle of Wight Studies, 1964–1974. Psychological Medicine. 1976;6(02):313.
3. Lenhard A, Lenhard W, Suggate S, Segerer R. A continuous solution to the norming problem. Assessment. 2016Feb;25(1):112–25. doi: 10.1177/1073191116656437 27371826
4. Oosterhuis HEM, van der Ark LA, Sijtsma K. Standard errors and confidence intervals of norm statistics for educational and psychological Tests. Psychometrika. September 2017;82(3):559–88.
5. Zhu J, Chen H-Y. Utility of inferential norming with smaller sample sizes. Journal of Psychoeducational Assessment. December 2011;29(6):570–80.
6. Koenker R. Quantreg: An R package for quantile regression and related methods, 2004. Available from:http://cran.r-project.org.
7. Wei Y, Pere A, Koenker R, He X. Quantile regression methods for reference growth charts. Statistics in Medicine. 30. April 2006;25(8):1369–82. doi: 10.1002/sim.2271 16143984
8. Wechsler DA. Wechsler adult intelligence scale-revised (WAIS-R). San Antonio, Texas: The Psychological Corporation; 1981.
9. Wechsler DA. Wechsler adult intelligence scale-third edition (WAIS-III). San Antonio, Texas: The Psychological Corporation; 1997.
10. Wechsler DA. Wechsler adult intelligence scale-fourth edition (WAIS-IV). San Antonio, Texas: Pearson; 2008.
11. Wechsler DA. The Wechsler intelligence scale for children—third edition (WISC-III). San Antonio, Texas: The Psychological Corporation; 1991.
12. Wechsler DA. The Wechsler intelligence scale for children—fourth edition (WISC-IV). San Antonio, Texas: Pearson; 2003.
13. Wechsler DA. The Wechsler intelligence scale for children—fifth edition (WISC-V). San Antonio, Texas: Pearson; 2014.
14. Kaufman A, Kaufman N. Kaufman Assessment Battery for Children, Second Edition (KABC-II). San Antonio, Texas: Pearson; 2004.
15. Zachary RA, Gorsuch RL. Continuous norming: implications for the WAIS-R. Journal of Clinical Psychology. January 1985;41(1):86–94. doi: 10.1002/1097-4679(198501)41:1<86::aid-jclp2270410115>3.0.co;2-w 3973045
16. Cole TJ, Green PJ. Smoothing reference centile curves: The lms method and penalized likelihood. Statistics in Medicine. 1992;11(10):1305–19. doi: 10.1002/sim.4780111005 1518992
17. Cole T. Fitting smoothed centile curves to Reference Data. Journal of the Royal Statistical Society Series A (Statistics in Society). 1988;151(3):385.
18. Rigby RA, Stasinopoulos DM. Smooth centile curves for skew and kurtotic data modelled using the Box–Cox power exponential distribution. Statistics in Medicine. 2004Jan;23(19):3053–76. doi: 10.1002/sim.1861 15351960
19. Rigby RA, Stasinopoulos DM. Generalized additive models for location, scale and shape (with discussion). Journal of the Royal Statistical Society: Series C (Applied Statistics). 2005;54(3):507–54.
20. Rigby R, Stasinopoulos D. Using the Box-Cox t distribution in GAMLSS to model skewness and kurtosis. Statistical Modeling: An International Journal. 2006;6(3):209–229.
21. Stasinopoulos DM, Rigby RA, Heller GZ, Voudouris V, De Bastiani F. Flexible Regression and Smoothing–Using GAMLSS in R. Boca Raton: CRC Press; 2017.
22. Lenhard A, Lenhard W, Gary S. cNORM—generating continuous test norms [document on the internet]. Dettelbach/Germany: Psychometrica; 2018. Available from: https://www.psychometrica.de/cNorm_en.html
23. Lenhard W, Lenhard A, Gary S. Continuous norming (version 1.1.12) [software repository on the internet]; 2018. Available from: https://github.com/WLenhard/cNORM
24. Stasinopoulos DM, Rigby RA, Voudouris V, Akantziliotou C, Enea M, Kiose D. Package ‘gamlss’ (Version 5.1‒2). Cran-Repository; 2018Oct06.
25. Lenhard W, Schneider W. ELFE 1–6: ein Leseverständnistest für Erst- bis Sechstklässler. Göttingen: Hogrefe; 2006.
26. Lenhard A, Lenhard W, Segerer R, Suggate S. Peabody picture vocabulary test—4. Ausgabe (German Adaption). Frankfurt am Main: Pearson; 2015.
27. Lenhard W, Lenhard A, Schneider W. ELFE II: ein Leseverständnistest für Erst- bis Siebtklässler—Version II. Göttingen: Hogrefe; 2017.
28. Stasinopoulos DM. GAMLSS practicals for the Graz short course. document on the internet]. 2016Nov05; Available from:http://www.stat.tugraz.at/friedl/GAMLSS/Practical-Graz.pdf
29. Grob A, Hagmann-von Arx P. IDS-2: Intelligenz- und Entwicklungsskalen für Kinder und Jugendliche. Bern: Hogrefe; 2018
30. Voncken L, Albers CJ, Timmerman ME. Improving confidence intervals for normed test scores: Include uncertainty due to sampling variability. Behavior Research Methods [document on the internet]. 2018Nov06; Available from: http://link.springer.com/10.3758/s13428-018-1122-8
Článek vyšel v časopise
PLOS One
2019 Číslo 9
- Případ Bulovka: Jak postupují jiné nemocnice, aby podobným tragédiím zabránily?
- Jak často se u masových střelců vyskytují duševní choroby?
- FDA varuje před selfmonitoringem cukru pomocí chytrých hodinek. Jak je to v Česku?
- Metamizol jako analgetikum první volby: kdy, pro koho, jak a proč?
- Klinický význam menších rozestupů mezi silami přípravků s levothyroxinem v léčbě hypotyreózy
Nejčtenější v tomto čísle
- Graviola (Annona muricata) attenuates behavioural alterations and testicular oxidative stress induced by streptozotocin in diabetic rats
- CH(II), a cerebroprotein hydrolysate, exhibits potential neuro-protective effect on Alzheimer’s disease
- Comparison between Aptima Assays (Hologic) and the Allplex STI Essential Assay (Seegene) for the diagnosis of Sexually transmitted infections
- Assessment of glucose-6-phosphate dehydrogenase activity using CareStart G6PD rapid diagnostic test and associated genetic variants in Plasmodium vivax malaria endemic setting in Mauritania
Zvyšte si kvalifikaci online z pohodlí domova
Všechny kurzy