Awareness of treatment: A source of bias in subjective grading of ocular complications

Autoři: Genis Cardona aff001;  Noelia Esterich aff001
Působiště autorů: Optics and Optometry Department, Universitat Politècnica de Catalunya, Terrassa, Spain aff001
Vyšlo v časopise: PLoS ONE 14(12)
Kategorie: Research Article



Bias has been described as one important obstacle in scientific research. The aim of this study was to explore “awareness of treatment” as a possible source of bias in subjective grading of ocular complications.


Thirty subjects with similar, basic experience with grading scales participated in the study. The Efron grading scales were used to grade 24 images of three different ocular conditions (eight images each of bulbar hyperaemia, limbal vascularization and corneal staining). Three consecutive, two weeks apart, grading sessions were scheduled, in which the same images were graded, although in the third session images were deceptively labelled as either “treated” or “untreated”. Grading results from the first and second sessions were compared to determine grading reliability and discrepancies with the third session informed of grading bias originating from “awareness of treatment”.


Moderate to good test-retest reliability was found for all conditions, with median intraclass correlation values of 0.80 (0.62–0.84) for bulbar hyperaemia, 0.68 (0.65–0.77) for limbal vascularization and 0.68 (0.66–0.74) for corneal staining. Grading values from the first and third sessions evidenced negative and positive systematic errors (bias) for “treated” and “untreated” conditions, respectively. Statistically significant differences were found between the average grading discrepancies of session 1 and session 2 and those of session 1 and session 3 (all p<0.001).


“Awareness of treatment” may be considered a source of bias of subjective grading of ocular complications, although the actual effect of bias is unlikely to be of clinical significance.

Klíčová slova:

Cornea – Eye lens – Eyes – Health education and awareness – Optical lenses – Research validity – Statistical data – Lens disorders


1. Efron N, Pritchard N, Brandon K, Copeland J, Godfrey R, Hamlyn B, et al. A survey of the use of grading scales for contact lens complications in optometric practice. Clin Exp Optom. 2011; 94:193–199. doi: 10.1111/j.1444-0938.2010.00549.x 21175821

2. Efron N: Efron grading scales for contact lens complications (Appendix K), in Efron N(ed): Contact Lens Practice, 2nd ed. Oxford, Butterworth-Heinemann-Elsevier, 2010, p 459–461.

3. Institute for Eye Research Grading Scales (Appendix B), in Phillips A, Speedwell L (eds): Contact Lenses, 5th ed. Oxford, Butterworth-Heinemann-Elsevier, 2007, p 627–631.

4. Wolffsohn JS, Naroo SA, Christie C, Morris J, Conway R, Maldonado-Codina C, et al. Anterior eye health recording. Contact Lens Ant Eye. 2015;38:266–271.

5. Frost N, Sparrow J. The assessment of lens opacities in clinical practice: results of a national survey. Br J Ophthalmol. 2001;85:319–321. doi: 10.1136/bjo.85.3.319 11222338

6. Troutbeck R, Hirst L. Review of treatment of pterygium in Queensland: 10 years after a primary survey. Clin Exp Ophthalmol. 2001;29:286–290. doi: 10.1046/j.1442-9071.2001.00435.x 11720153

7. Kunnen CME, Wolffsohn JS, Ritchey ER. Comparison of subjective grading lid wiper epitheliopathy with a semi-objective method. Contact Lens Ant Eye. 2019.

8. Woods J, Varikooty J, Fonn D, Jones LW. A novel scale for describing corneal staining. Clin Ophthalmol. 2018;12:2369–2375. doi: 10.2147/OPTH.S178113 30538419

9. Begley C, Caffery B, Chalmers R, Situ P, Simpson T, Nelson JD. Review and analysis of grading scales for ocular surface staining. Ocul Surf. 2019.

10. Arita R, Minoura I, Morishige N, Shirakawa R, Fukuoka S, Asai K, et al. Development of definitive and reliable grading scales for Meibomian gland dysfunction. Am J Ophthalmol. 2016;169:125–137. doi: 10.1016/j.ajo.2016.06.025 27345733

11. Downie LE, Keller PR, Vingrys AJ. Assessing ocular bulbar redness: a comparison of methods. Ophthalmic Physiol Opt. 2016;36:132–139. doi: 10.1111/opo.12245 26890702

12. Park IK, Chun YS, Kim KG, Yang HK, Hwang J-M. New clinical grading scales and objective measurement for conjunctival injection. Invest Ophthalmol Vis Sci. 2013;54:5249–5257. doi: 10.1167/iovs.12-10678 23833063

13. Chun YS, Yoon WB, Kim KG, Park IK. Objective assessment of corneal staining using digital image analysis. Invest Ophthalmol Vis Sci. 2014;55:7896–7903. doi: 10.1167/iovs.14-15618 25406292

14. Sorbara L, Peterson R, Schneider S, Woods C. Comparison between live and photograph slit lamp grading of corneal staining. Optom Vis Sci. 2015;92:312–317. doi: 10.1097/OPX.0000000000000496 25546827

15. Efron N, Morgan PB, Jagpal R. The combined influence of knowledge, training and experience when grading contact lens complications. Ophthalmic Physiol Opt. 2003;23:79–85. doi: 10.1046/j.1475-1313.2003.00091.x 12535060

16. Andersen F, Anjum RL, Rocca E. Philosophical bias is the one bias that science cannot avoit. Elife. 2019;8:e44929. doi: 10.7554/eLife.44929 30864947

17. Catalogue of Bias Collaboration. Mahtani K, Spencer EA, Brassey J. Observer Bias. In Catalogue of Bias 2017. Available from:

18. Bailey IL, Bullimore MA, Raasch TW, Taylor HR. Clinical grading and the effects of scaling. Invest Ophthalmol Vis Sci. 1991;32:422–432. 1993595

19. Catalogue of Bias Collaboration. Mahtani K, Spencer EA, Brassey J. Misclassification bias. In Catalogue of Bias 2017. Available from:

20. Schmidt RL, Factor RE. Understanding source of bias in diagnostic accuracy studies. Arch Pathol Lab Med. 2013;137:558–565. doi: 10.5858/arpa.2012-0198-RA 23544945

21. Efron N. Grading scales for contact lens complications. Ophthalmic Physiol Opt. 1998;18:182–186. doi: 10.1016/s0275-5408(97)00066-5 9692040

22. Scott J, Huskisson EC. Graphic representation of pain. Pain. 1976;2:75–84.

23. Carkeet A. Exact parametric confidence intervals for Bland-Altman limits of agreement. Optom Vis Sci. 2015;92:e71–e80. doi: 10.1097/OPX.0000000000000513 25650900

24. Bland JM, Altman DG. Measuring agreement in method comparison studies. Stat Methods Med Res. 1999;8:135–160. doi: 10.1177/096228029900800204 10501650

25. Efron N, McCubbin S. Grading contact lens complications under time constraints. Optom Vis Sci. 2007;84:1082–1086. doi: 10.1097/OPX.0b013e31815b9dfc 18091306

26. Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med. 2016;15:155–163. doi: 10.1016/j.jcm.2016.02.012 27330520

27. Liljequist D, Elfving B, Roaldsen KS. Intraclass correlation–A discussion and demonstration of basic features. PLoS One. 2019;14:e0219854. doi: 10.1371/journal.pone.0219854 31329615

28. Wu S, Hong J, Tian L, Cui X, Sun X, Xu J. Assessment of bulbar redness with a newly developed Keratograph. Optom Vis Sci. 2015;92:892–899. doi: 10.1097/OPX.0000000000000643 26099055

29. Chong T, Simpson T, Fonn D. The repeatability of discrete and continuous anterior segment grading scales. Optom Vis Sci. 2000;77:244–251. doi: 10.1097/00006324-200005000-00011 10831214

Článek vyšel v časopise


2019 Číslo 12
Nejčtenější tento týden