The psychometric validation of the Dutch version of the Rivermead Post-Concussion Symptoms Questionnaire (RPQ) after traumatic brain injury (TBI)
Anne Marie Plass aff001; Dominique Van Praag aff002; Amra Covic aff001; Anastasia Gorbunova aff001; Ruben Real aff001; Nicole von Steinbuechel aff001;
Authors place of work:
Institute of Medical Psychology and Medical Sociology, University Medical Center Göttingen (UMG)/ Georg-August-University, Göttingen, Germany
aff001; Department of Neurosurgery, Antwerp University Hospital and University of Antwerp, Edegem, Belgium
Published in the journal:
PLoS ONE 14(10)
Traumatic brain injury (TBI) is one of the most common neurological conditions. It can have wide-ranging physical, cognitive and psychosocial effects. Most people recover within weeks to months after the injury, but a substantial proportion are at risk of developing lasting post-concussion symptoms. The Rivermead Post-Concussion Syndrome Questionnaire (RPQ) is a short validated 16-items self-report instrument to evaluate post-concussive symptoms. The aim of this study was to test psychometrics characteristics of the current Dutch translation of the RPQ.
To determine the psychometric characteristics of the Dutch RPQ, 472 consecutive patients six months after they presented with a traumatic brain injury in seven medical centers in the Netherlands (N = 397), and in two in Belgium (Flanders) (N = 75) took part in the study which is part of the large prospective longitudinal observational CENTER-TBI-EU-study. Psychometric properties at six months post TBI, were assessed using exploratory and confirmatory factor analyses. Sensitivity was analyzed by comparing RPQ scores and self-reported recovery status of patients with mild vs. moderate and severe TBI.
The Dutch version of RPQ proved good, showing excellent psychometric characteristics: high internal consistency (Cronbach’s α .93), and good construct validity, being sensitive to self-reported recovery status at six months post TBI. Moreover, data showed a good fit to the three dimensional structure of separate cognitive, emotional and somatic factors (Chi2 = 119; df = 117; p = .4; CFI = .99; RMSEA = .006), reported earlier in the literature.
Psychometric characteristics of the Dutch version of RPQ proved excellent to good, and can the instrument therefore be applied for research purposes and in daily clinical practice.
Traumatic brain injury (TBI) is one of the most common neurological conditions, and occurs when an external force causes brain trauma . TBI can be classified as mild, moderate or severe, and may have wide-ranging physical and psychological effects [2–7]. Some signs or symptoms appear immediately after the traumatic event, while others days or weeks later [3,5]. In the Netherlands, about 85,000 people are confronted with a traumatic brain injury, on a yearly basis . As not everyone confronted with TBI seeks professional help, this number might be an underestimation . On average, 30,000 of these seek help at the Emergency Room (ER) of the hospital, and about 21,000 require hospital stays . Yearly, about 1,000 die because of TBI . Most people suffer from mild TBI (mTBI), e.g. concussion, for which they often do not seek professional help, or seek advice from their General Practitioner (GP): In the Netherlands, the GP functions as gatekeeper to the rest of the medical system. . Virtually all non-institutionalized Dutch citizens are registered with a GP controlling access to specialized medical care. 
In about one third of the cases mTBI leads to long-term consequences [5,12]. A substantial proportion (about 15–30%) of individuals after mTBI are at risk to developing post-concussion symptoms . These can be classified into four categories: cognitive difficulties (e.g. concentration and memory loss), behavioral maladaptation (e.g. impulsivity, and aggressive behavior), psychiatric conditions (e.g. posttraumatic stress), and physical disorders (e.g. chronic pain) . Whether patients develop post-concussion symptoms, is associated with cognitive, emotional, behavioral and social risk factors, and does not necessarily depend on the severity of the traumatic brain injury, see Fig 1 . Years of education, pre-injury psychiatric disorders, neck pain and prior TBI were found strong predictors of 6-month post-concussive symptoms [3,5], as were patient’s perceptions of their brain injury, their behavioral responses, passive and avoidant coping styles and emotional distress in response to this [2,5].
As most people recover from their TBI within weeks to months after the injury, post-concussion symptoms might easily be overlooked, since these residual complaints may be deferred. Furthermore, imaging techniques often do not show any structural brain damage in this population [13,14]. Still, one in three mTBI patients will not be able to resume work and activities six months after the event at a level similar to that before the accident . As a consequence, (m)TBI is associated with substantial ongoing disability and distress for patients, and high healthcare costs [2,9]. A possible instrument for (early) identification in order to timely guide clinical management of post-concussion symptoms after TBI, is the Rivermead Post Concussion Syndrome Questionnaire (RPQ). The RPQ is a validated measurement-instrument to survey post-concussion symptoms, relying on self-report as to the presence and severity of 16 symptoms [15–18]. The items form one scale, but were not always found to tap into the same underlying construct [3,4,14]. Eyres et al (2005) found no evidence for a single factor structure, and proposed to split the RPQ into two subscales consisting of the first three items ‘RPQ3’, representing immediate symptoms (headaches, dizziness, and nausea) and the remaining 13 items ‘RPQ13’, representing symptoms that might occur at a later stage . On the other hand, Lannsjö et al (2011) found strong support for both a single and two factor structure in their RPQ validation study, but failed to reproduce the RPQ3/13 two-category model as suggested by Eyres et al (2005). Furthermore, a ‘rationally-based’ three categories model was proposed by Smith-Seemiller and colleagues (2003), comprising of the following domains: 1. cognitive deficits (impaired memory, poor concentration, slow thinking), 2. somatic complaints (headaches, dizziness, nausea, blurred or double vision, noise or light sensitivity, sleep disturbance, fatigue), and 3. emotional complaints (irritably, depression, frustration, restlessness), serving as framework in various studies on post-concussion symptoms [3,19]. The results of Potter and colleagues (2006) supported the existence of separate cognitive, emotional and somatic factors . So far, the RPQ has been validated in various languages [4,6], but until now this has not been the case for the Dutch version of this questionnaire. Therefore, this study aims to investigate the psychometric characteristics of the current Dutch translation of the RPQ.
This study is part of the Collaborative European NeuroTrauma Effectiveness Research in TBI (CENTER-TBI) study, which is a prospective longitudinal observational study conducted in 72 centers from 21 countries . In the Netherlands, patients were recruited from seven medical centers spread over the country: Leiden University Medical Center (LUMC), University Medical Center Groningen (UMCG), Erasmus MC Rotterdam, Radboud University medical center Nijmegen, Medical center Haaglanden The Hague, Elisabeth Hospital Tilburg, HAGA hospitals The Hague. Furthermore, two centers in the Dutch-speaking-part of Belgium (Flanders) were included in the study: the Antwerp University Hospital, and the University Hospital in Leuven. Patients that presented within 24 hours after brain injury at the hospital, that were diagnosed with TBI, and had a clinical indication for CT scan, were eligible for the study, and were all invited to participate in this convenience sample. Those willing to participate (N = 822) provided written informed consent prior to inclusion, either personally, or through a legally designated representative. Participants were free to withdraw at any point in time in the study without stating a reason . Patients with severe pre-existing neurological disorder, i.e. Cerebrovascular accident, Transient ischemic attacks, and Epilepsy that could confound the outcome assessment, were excluded. A written informed consent to participate in the study was obtained at the time of inclusion. At six months post TBI, the nurse at the center administered the RPQ during a visit, or was sent by postal mail to those who did not need to attend the hospital, for completion at home. A pre-franked envelope was included to send it back.
Translation of the Dutch RPQ
Two native Dutch speakers who are proficient in English translated the RPQ into Dutch. Subsequently, they compared their translations and discussed differences until mutual agreement had reached. As a next step, a native English speaker, who is fluent in Dutch, backward translated this harmonized version. This version was then compared to the original English RPQ version possible differences were identified and resolved by the two parties. In addition, a team of researchers and CENTER-TBI collaborators refined and reshaped the measurement-instrument until consensus was reached, following an iterative process. This multiple-step procedure resulted in a final version of the Dutch RPQ.
The CENTER-TBI study has been conducted in accordance with all relevant laws of the EU if directly applicable or of direct effect, and all relevant laws of the country where the Recruiting sites were located, including, but not limited to, the relevant privacy and data protection laws and regulations (the “Privacy Law”), the relevant laws and regulations on the use of human materials, and all relevant guidance relating to clinical studies from time to time in force including, but not limited to, the ICH Harmonised Tripartite Guideline for Good Clinical Practice (CPMP/ICH/135/95) (“ICH GCP”) and the World Medical Association Declaration of Helsinki entitled “Ethical Principles for Medical Research Involving Human Subjects”. Ethical approval was obtained for each recruiting site. Informed Consent was obtained for all patients recruited in the Core Dataset of CENTER-TBI and documented in the e-CRF. The list of sites, Ethical Committees, approval numbers and approval dates can be found on the official Center TBI website (www.center-tbi.eu/project/ethical-approval).
The Rivermead Post-Concussion Questionnaire (RPQ) consists of 16 common symptoms related to post concussion. Patients are asked to rate how problematic symptoms were compared to the situation before the head injury on a 5-point Likert scale (0–4). A score of 0 indicating ‘not experienced at all; 1 indicating ‘no more of a problem (than before)’, 2 indicating ‘a mild problem’; 3 indicating ‘a moderate problem; 4 indicating ‘a severe problem’ . Scores are taken as sum of all symptom scores, excluding scores of 1, as these indicate symptoms are unchanged since the brain injury. This gives a potential total score range of 0 (representing no change in symptoms since the head injury) to 64 (most severe symptoms) . If more than 5 of the items were missing from the 16, a score was not calculated and treated as missing. The RPQ total score is calculated using prorating as imputation method, if up to one third of the items were missing. In addition, the RPQ scoring method of Stulemeijer et. al. (2008) was applied where a score of highest 2 (a mild problem) to at least 13 of the 16 items is defined a favorable outcome. Stulemeijer et al (2008) showed that 94% of non-brain-injured patients (wrist-, or ankle distortion) would meet this criterion .
Further, TBI severity was rated using the Glascow Coma Scale (GCS), with scores between 3–8 indicating severe, 9–12 moderate, and 13–15 mild TBI [21–25]. The GCS was administered within the first 24 hours after the brain injury occurred. Current disability was assessed by administering the extended Glasgow Outcome Scale (GOSE) . GOSE scores were used to differentiate between patients with remaining severe disability (3–4), moderate disability (5–6), and good recovery (7–8) [27,21]. In addition, socio- demographic data were collected, including gender, age, working status, education level, etc.
The psychometric characteristics of the Dutch version of the RPQ were determined at six months post TBI, using SPSS version 24, AMOS version 24, and R version 3.3.3, amongst which. the lavaan-package  to performing analyses on the factor structure. Data extraction was based on the core CENTER-TBI dataset from November 2017.
Internal consistency was determined by calculating Cronbach’s alpha, including alpha if any item was deleted. To testing construct validity, Principal Axis Factoring (PAF) was done by unweighted least squares and oblimin rotation on the 16 RPQ items, exploring the underlying constructs in the Dutch version of the RPQ, as no consistent underlying factor structure has been established so far. Items were included if the factor loading was 0.5 or higher and if factor loadings on the other factors were 0.1 or lower. Confirmatory factor analysis (CFA) was used to examine the fit of the Dutch RPQ data to the various factor structures that had been described earlier in literature: For this, we used the single model factor, reflecting post-concussion symptoms as unitary entity ; the RPQ3 and RPQ13 two factor model ; and the three factor model [18,19]. The goodness of the fit was based on CFI values above .95 and RMSEA values below .06 indicating good fit [29,30]
Concurrent criterion validity was assessed by analyzing the influence of important covariates on RPQ scores (GCS, GOSE) using both t-tests or one-way ANOVA, and non-parametric Wilcoxon Rank Sum W tests for independent samples. The level of significance used was .05, and .016 (padj = .05/3 = .016) for Bonferroni adjusted multiple comparisons (i.e., pairwise independent Wilcoxon rank sum tests) Descriptive analyses were performed for sociodemographic variables (gender, age, education level, etc.). Although people in The Netherlands and Flanders (Belgium) both speak Dutch, the language used differs, and words might have a different meaning. Therefore, all tests were performed both for the entire research sample and for each country separately where possible.
In total 472 patients filled in the Dutch version of the RPQ at six months post TBI (Response Rate (RR) = 57%). Of these, 397 were administered in the Netherlands and 75 in Belgium. Twenty-five participants were aged under 18 (18 in the Netherlands, and seven in Belgium) and were excluded from this study. Country of residence was registered for 437 patients, who were either living in Belgium (N = 67), or the Netherlands (N = 368), apart from two respondents who reported Nepal for country of residence. Not all respondents were born in the Netherlands or Belgium (see Table 1), but since their understanding of the Dutch language was sufficient to fill in the RPQ, and since they currently were living in the Netherlands or Flanders, they were not excluded from the study. There were 277/ 447 (62%) male respondents (resp. 231/ 379 (60.9%) in the Netherlands, and 46/ 68 (67.6%) in Belgium). The vast majority of the study population (68.2%) belonged to the middle aged and older age groups (38% was aged 45–65, 30.2% 65 and up). More than half were higher educated reporting having completed college or university (64.2%), and were either married or living together (56.6%). Nearly half were not (e.g. students, unemployed people), or no longer (e.g. retired) employed (48.4%), see Table 2 for further details. There were no significant differences in total RPQ scores at six months post TBI for study participants in the Netherlands (M = 12.63; SD = 13.77) and participants in Belgium (M = 12.64; SD = 11.94; t(445) = -110; p = .9; W = 84271.5; p = .5). The magnitude of the differences in the means was very small (eta squared < .0001). No significant difference in RPQ total score were found between the center administered and postal questionnaire modes (W = 15390, p = .219)’. The majority of 71.1% (N = 349) filled in and returned the postal questionnaire. Another 21.5% (N = 96) responded to the questionnaire at the time they visited the hospital, and two participants with reading difficulties responded through a phone interview.
At the time of the injury, 13.4% (N = 60) of the study population solely attended the ER without further hospitalization. 52.6% (N = 235) were hospitalized, and 34% (N = 152) needed a stay in the ICU. Initially, within 24 hours after TBI, 80.5% (N = 316) were diagnosed mTBI, 5.8% (N = 21) were diagnosed moderate TBI, and 6.4% (N = 23) with severe TBI. Of 87 respondents (19.5%) these data were missing. Six months after the brain injury, GOSE scores reveal that 55.5% (N = 248) of the respondents showed good recovery, 25% (N = 112) reported moderate disabilities, and 6.7% (N = 30) suffered from severe disabilities at that point in time. Of 57 respondents (12.8%) these data were missing. Those patients that solely attended the ER without further hospitalization, all were initially diagnosed mTBI (see Table 3). Of 313 respondents all three data types were available (hospitalization, initial diagnosis and six months post recovery status), see Table 4 for further descriptives.
Prior to performing PAF the suitability of data for factor analyses was assessed. Inspection of the correlation matrix revealed the presence of many coefficients of .3 and above. The Kaiser-Meyer-Oklin (KMO) value was .94, exceeding the recommended value of .6 (Kaiser 1970, 1974) and the Barlett’s Test of sphericity (Barlett, 1954) reached statistical significance (p <.001), supporting the factorability of the correlation matrix. PAF, using Oblimin rotation, revealed the presence of three components with eigenvalue exceeding 1, explaining 47.7%, 5.0%, and 4.0% of the variance respectively. The scree plot revealed a clear break after the first component, see Fig 2. Moreover, all items except for three (nausea (.42), blurred vision (.42) and double vision (.34)) show factor loadings of .5 and up on the first factor, but high factor loadings (>.1) on at least one of the other factors too. Confirmatory Factor analysis (CFA) was run to testing model fit to possible underlying factor structures that had been described in literature earlier [4,6,18,19]. A central assumption is that the data are distributed normally. However, substantial problems with univariate skew and kurtosis were identified, see Table 4. Given the (expected) highly unequal distribution of response categories across items (0: 56%, 1: 17%, 2: 14%, 3: 9%, 4: 4%), responses to categories 0 (‘not experienced at all’) and 1 (‘no more of a problem’), as well as responses to categories 2 (‘mild problem’), 3 (‘moderate problem’) and 4 (‘severe problem’) were collapsed into new categories ‘no problem`and ‘some problems’ and the robust WLSMV estimator was used for CFA . Following this, CFA indicated a lack of fit to unitary model, given the significant Chi-squares , and a lack of fit to the RPQ3/ RPQ13 two component model , but showed a good fit to the three factor structure , see Table 5. The Belgian sample was too small to perform separate CFAs.
The RPQ showed high consistency with Cronbach’s alpha being .93. For the sample in the Netherlands Cronbach’s alpha was .94, and for the sample in Belgium Cronbach’s was alpha .91. The scale did not improve if any items were deleted. Spearman Brown Coefficient rsb1 was .91). Further, item characteristics showed high item correlations (>.55, except for double vision and nausea), and acceptable asymmetry for all items but double vision (skewness being 2.6), see Table 6.
Concurrent criterion validity
RPQ total scores at six months post TBI of patients (self-)reporting severe (M = 20.7; SD = 18.3; N = 30) or moderate (M = 20.2; SD = 13.9; N = 112) disabilities according to their total scores on the GOSE scale, differed significantly to those that showed good recovery at this point in time (M = 8.3; SD = 10.6; N = 248) (F(2, 387) = 42.7; p <.001). RPQ total scores at six months post TBI further were found to differentiate between initial mTBI (M = 11.6; SD = 13.2; N = 316) and moderate TBI (M = 20.2; SD = 16.8; N = 21) diagnoses (GCS-scores) (F(2, 357) = 4.5; p = .012). Remarkably, RPQ total scores of those initially diagnosed with severe TBI (M = 14.8, SD = 13.9; N = 23), resembled most those initially diagnosed with mTBI (NS). When recalculating RPQ total score into favorable (a score of highest 2 to at least 13 of the 16 items  vs unfavorable, 74.7% (N = 334) of the study population had a favorable outcome at six months post TBI, indicating that the symptoms reported, do not differ from what can be found in a non-TBI population [29,30]. 25.3% (N = 113) still had an unfavorable, strongly related to TBI, outcome. The RPQ score was found to discriminate between recovery status (GOSE scores) at six months post TBI (Chi2 = 45.2; df = 2; p <.001), although not strongly (Cramer’s V = .11).
When solely taking the sample from the Netherlands into account, a stronger relationship between favorable and unfavorable RPQ outcomes and recovery status (GOSE scores) at six month post TBI was found (Cramer’s V = .35, Chi2 = 43.8; df = 2; p <.001). Further, the RPQ total scores at six months post TBI of the Dutch sample were found to discriminate between recovery status (GOSE scores) at six months post TBI (F(2, 358) = 39.2; p <.001; KH(2) = 70.62; p <.001), and initial diagnoses within 24 hours after the brain injury occurred (GCS Scores) (F(2, 299) = 3.7; p = .03; KH(2) = 8.172; p = .02). The number of participants from Belgium that could be included in these analyses were too low for further analyses.
The current Dutch translation of the RPQ showed good psychometric characteristics, with high internal consistency, and good construct validity, consistent with the findings of others. As for these aspects, it can be applied for research purposes and in daily clinical practice, as an instrument to identify post-concussion symptoms. Besides, it proved sensitive for recovery status at six months post TBI, showing that those who (self-) reported moderate or severe disabilities six months after the brain injury took place, had significant lower RPQ total scores compared to those reporting good recovery at that time point. However, the latter needs to be treated with caution, as the numbers of patients with moderate and severe TBI in this study were rather low. Further, RPQ total scores at six months post TBI were found to distinguish between initial TBI diagnoses: Those initially diagnosed with moderate TBI had higher RPQ total (sum) scores at six months post TBI compared to those initially diagnosed with mild TBI (mTBI). However, the number of people in our research sample that were initially diagnosed with moderate TBI was low. Moreover, RPQ total scores at six months post TBI of those initially diagnosed with severe TBI resembled more the total score of those diagnosed with mTBI, rather than those with moderate TBI. A possible explanation for this might be that moderate TBI and the amount of care needed was underestimated. This type of TBI might need more intensive care than what was provided. However, again the number of people in this group was too low to base further conclusions upon. Another explanation for this might be that the large number of mTBI-diagnosed patients who were admitted to the ICU, needed intensive care because of other injuries, and thus were diagnosed mTBI correctly.
Consistent with the findings of others, we found multidimensionality as underlying structure of the RPQ measurement-instrument [3,4,6,18,19,31,32]. However, high factor loadings of items on multiple factors, and the clear break after the first factor in the scree plot, would suggest a one factor structure rather than multidimensionality. Confirmative factor analyses on the other hand revealed that our data would fit best to the three-component model dividing the RPQ post-concussion symptoms into the following three categories: 1. cognitive deficits (impaired memory, poor concentration, slow thinking), 2. somatic complaints (headaches, dizziness, nausea, blurred or double vision, noise or light sensitivity, sleep disturbances), and 3. emotional complaints (irritably, depression, frustration, restlessness). Small differences were found between the Belgium and Dutch sample, with only the Dutch sample showing a good fit to this model. However the number of respondents in the Belgium sample was below 250, due to which the criteria for model fit may not be valid . The variation in underlying structure of the RPQ differs between studies and countries, and might be due to various reasons. One reason could be the convenience sample used for this research. Another explanation might be the different analyses techniques used in the various studies, as modern techniques often tend to disqualify measurement-instrument validity that had been established before by classical analyses methods . A third possible explanation underpinning this may be the way in which measurement-instruments are being translated in accordance to the WHO guidelines of forward and back translation . In order not to lose the potential to comparing data, researchers prefer to stay as close as possible to the original version. However, through this, the principles of cultural interpretation and translating the correct meaning of the items, might be missed out, due to which country differences might occur, even though the instrument used is very similar. [36,37].
In addition, we argue that despite the underlying multidimensionality found, the Dutch version of the RPQ needs not necessarily be divided into subscales when applied for research purposes and in daily clinical practice. The underlying multidimensionality might indicate that post-concussion symptoms represent more than one dimension, but factors highly correlate, and items were not unique for just one of the factors. Moreover, there is a large body of evidence that supports the use of total scores of scales to which multidimensionality is a precondition, e.g. attitude scales that usually exist of a cognitive and affective component . Furthermore, as the psychometric properties of the Dutch version of the RPQ proved good, it would be of interest to implement this measurement-instrument into primary care settings, in order to timely recognize the possible long-term consequences of TBI. This would especially be effective in countries as the Netherlands where one has to see the GP first, before entering the rest of the medical system, the so-called gatekeeper system, and in ER settings, where people usually are only checked medically and then send home, in order to timely identify potential patients at risk. However, more clarity is needed on how to best interpret self-reported RPQ scores , since similar symptoms can too be reported by those suffering from different injuries and, disorders, or by members of the general population as fatigue, headaches, nausea etc., are very common. As such, Stulemeijer and colleague (2016) found that 94% of non-TBI patients with wrist or ankle distortion too score positive on a maximum of three RPQ items. Furthermore, self-report might not always be the best indicator for reporting symptoms, however in the case of non-social desirable issues, such as disease burden and the report of minor ailments it has been shown valid [39–41].
At six months post TBI, three quarters of the research sample no longer showed post-concussion symptoms, which is consistent with findings elsewhere that in about 15–30% of the cases mTBI leads to long-term consequences [2, 5,12], due to which data were skewed and not normally distributed. Validation of the RPQ at this point in time might therefore be difficult. Further, items were strongly correlated, due to which items strongly loaded on one and the same factor. Another limitation of this study was the limited Belgian sample, which was often too small for sound complex analyses, such as CFA. Further, the lacking of a construct validation phase making use of cognitive interviewing limits the overall conclusion on validity of the Dutch version of the RPQ , especially since there were between-country differences. Knowing what our respondents think we are asking, and knowing how they interpret the questions we are asking might help to explain the variety in underlying symptom structure found too . Moreover, the convenience hospital sample used in this study might be limited representative to the entire mTBI population, as most people in the Netherlands tend not to seek specialized medical help for their head injury. Last, as it is known that individuals with mild TBI are more likely to develop depression, and as depressed patients tend to report higher impairments , it should be taken into account in interpreting this data that patients might possibly report less severe symptoms at the time point of the brain damage, where depression might not be present yet.
The psychometric characteristics of the Dutch version of RPQ proved good, showing high consistency, and good construct validity, being sensitive to self-reported recovery status at six months post TBI and initial TBI-diagnosis sensitive. The Dutch version of the RPQ can therefore be applied for research purposes and in daily clinical practice. Further discussion is needed with regard to the scoring of the RPQ, as underlying multidimensionality may not necessarily stand in the way of using a total score.
1. Menon DK, Schwab K, Wright DW, Maas AI. Position Statement: Definition of Traumatic Brain Injury. Arch Phys Med Rehabil. 2010 Nov 1;91(11):1637–40. doi: 10.1016/j.apmr.2010.05.017 21044706
2. Hou R, Moss-Morris R, Peveler R, Mogg K, Bradley BP, Belli A. When a minor head injury results in enduring symptoms: a prospective investigation of risk factors for postconcussional syndrome after mild traumatic brain injury. J Neurol Neurosurg Psychiatry. 2012 Feb;83(2):217–23. doi: 10.1136/jnnp-2011-300767 22028384
3. Cnossen MC, Winkler EA, Yue JK, Okonkwo DO, Valadka A, Steyerberg EW, et al. Development of a Prediction Model for Post-Concussive Symptoms following Mild Traumatic Brain Injury: A TRACK-TBI Pilot Study. J Neurotrauma. 2017 Mar 27;
4. Eyres S, Carey A, Gilworth G, Neumann V, Tennant A. Construct validity and reliability of the Rivermead Post-Concussion Symptoms Questionnaire. Clin Rehabil. 2005 Dec;19(8):878–87. doi: 10.1191/0269215505cr905oa 16323387
5. van der Naalt J, Timmerman ME, de Koning ME, van der Horn HJ, Scheenen ME, Jacobs B, et al. Early predictors of outcome after mild traumatic brain injury (UPFRONT): an observational cohort study. Lancet Neurol. 2017 Jul;16(7):532–40.
6. Lannsjö M, Borg J, Björklund G, Af Geijerstam J-L, Lundgren-Nilsson A. Internal construct validity of the Rivermead Post-Concussion Symptoms Questionnaire. J Rehabil Med. 2011 Nov;43(11):997–1002. doi: 10.2340/16501977-0875 22031345
7. Majdan M, Plancikova D, Maas A, Polinder S, Feigin V, Theadom A, et al. Years of life lost due to traumatic brain injury in Europe: A cross-sectional analysis of 16 countries. PLoS Med. 2017 Jul;14(7):e1002331. doi: 10.1371/journal.pmed.1002331 28700588
8. Hersenstichting. Hersenstichting—Cijfers over patiënten [Internet]. Hersenstichting. [cited 2019 Mar 26]. https://www.hersenstichting.nl/alles-over-hersenen/hersenaandoeningen/cijfers-over-patienten
10. Zadok J. Numbers, causes, and consequences non-congenital brain damage. [In Dutch:’Aantallen, oorzaken en gevolgen niet-aangeboren hersenletsel’] [Internet]. Hersenz; 2015. http://www.hersenz.nl/sites/all/files/pdf/factsheet_aantallen_oorzaken_en_gevolgen_niet-aangeboren_hersenletsel.pdf
11. van den Dungen C, Hoeymans N, Gijsen R, van den Akker M, Boesten J, Brouwer H, et al. What factors explain the differences in morbidity estimations among general practice registration networks in the Netherlands? A first analysis. Eur J Gen Pract. 2008;14 Suppl 1:53–62.
13. Yuh EL, Mukherjee P, Lingsma HF, Yue JK, Ferguson AR, Gordon WA, et al. Magnetic resonance imaging improves 3-month outcome prediction in mild traumatic brain injury. Ann Neurol. 2013 Feb;73(2):224–35. doi: 10.1002/ana.23783 23224915
14. Yuh EL, Hawryluk GWJ, Manley GT. Imaging concussion: a review. Neurosurgery. 2014 Oct;75 Suppl 4:S50–63.
15. King NS. Emotional, neuropsychological, and organic factors: their use in the prediction of persisting postconcussion symptoms after moderate and mild head injuries. J Neurol Neurosurg Psychiatry. 1996 Jul;61(1):75–81. doi: 10.1136/jnnp.61.1.75 8676166
16. King NS, Crawford S, Wenden FJ, Moss NE, Wade DT. The Rivermead Post Concussion Symptoms Questionnaire: a measure of symptoms commonly experienced after head injury and its reliability. J Neurol. 1995 Sep;242(9):587–92. doi: 10.1007/bf00868811 8551320
17. King NS, Crawford S, Wenden FJ, Caldwell FE, Wade DT. Early prediction of persisting post-concussion symptoms following mild and moderate head injuries. Br J Clin Psychol. 1999 Mar;38 (Pt 1):15–25.
18. Potter S, Leigh E, Wade D, Fleminger S. The Rivermead Post Concussion Symptoms Questionnaire: a confirmatory factor analysis. J Neurol. 2006 Dec;253(12):1603–14. doi: 10.1007/s00415-006-0275-z 17063314
19. Smith-Seemiller L, Fow NR, Kant R, Franzen MD. Presence of post-concussion syndrome symptoms in patients with chronic pain vs mild traumatic brain injury. Brain Inj. 2003 Mar;17(3):199–206. 12623496
20. Maas AIR, Menon DK, Steyerberg EW, Citerio G, Lecky F, Manley GT, et al. Collaborative European NeuroTrauma Effectiveness Research in Traumatic Brain Injury (CENTER-TBI): a prospective longitudinal observational study. Neurosurgery. 2015 Jan;76(1):67–80. doi: 10.1227/NEU.0000000000000575 25525693
21. von Steinbüchel N, Wilson L, Gibbons H, Hawthorne G, Höfer S, Schmidt S, et al. Quality of Life after Brain Injury (QOLIBRI): scale validity and correlates of quality of life. J Neurotrauma. 2010 Jul;27(7):1157–65. doi: 10.1089/neu.2009.1077 20210602
22. Teasdale G, Maas A, Lecky F, Manley G, Stocchetti N, Murray G. The Glasgow Coma Scale at 40 years: standing the test of time. Lancet Neurol. 2014;13(8):844–854. doi: 10.1016/S1474-4422(14)70120-6 25030516
23. Teasdale G, Jennett B. Assessment of coma and impaired consciousness: a practical scale. The Lancet. 1974;304(7872):81–84.
24. Castaño-León AM, Navarro-Main B, Gomez PA, Gil A, Soler MD, Lagares A, et al. Quality of Life After Brain Injury: Psychometric Properties of the Spanish Translation of the QoLIBRI. Eval Health Prof. 2017;0163278717702696.
25. von Steinbüchel N, Real RGL, Sasse N, Wilson L, Otto C, Mullins R, et al. German validation of Quality of Life after Brain Injury (QOLIBRI) assessment and associated factors. PLOS ONE. 2017 May 24;12(5):e0176668. doi: 10.1371/journal.pone.0176668 28542226
26. Wilson JL, Pettigrew LE, Teasdale GM. Structured interviews for the Glasgow Outcome Scale and the extended Glasgow Outcome Scale: guidelines for their use. J Neurotrauma. 1998;15(8):573–585. doi: 10.1089/neu.1998.15.573 9726257
27. Stulemeijer M, Van der Werf S, Borm GF, Vos PE. Early prediction of favourable recovery 6 months after mild traumatic brain injury. J Neurol Neurosurg Psychiatry. 2008;79(8):936–942. doi: 10.1136/jnnp.2007.131250 17951281
28. Rosseel Y. lavaan: An R Package for Structural Equation Modeling. J Stat Softw [Internet]. 2012 [cited 2019 Jul 26];48(2). http://www.jstatsoft.org/v48/i02/
29. Brown TA. Confirmatory factor analysis for applied research. Guilford Publications; 2014.
30. Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struct Equ Model Multidiscip J. 1999 Jan;6(1):1–55.
31. Haagsma JA, Scholten AC, Andriessen TMJC, Vos PE, Van Beeck EF, Polinder S. Impact of depression and post-traumatic stress disorder on functional outcome and health-related quality of life of patients with mild traumatic brain injury. J Neurotrauma. 2015 Jun 1;32(11):853–62. doi: 10.1089/neu.2013.3283 25320845
32. Cnossen MC, Scholten AC, Lingsma HF, Synnot A, Haagsma J, Steyerberg PEW, et al. Predictors of Major Depression and Posttraumatic Stress Disorder Following Traumatic Brain Injury: A Systematic Review and Meta-Analysis. J Neuropsychiatry Clin Neurosci. 2017;29(3):206–24. doi: 10.1176/appi.neuropsych.16090165 28193126
33. Hu LT, Bentler PM, Kano Y. Can test statistics in covariance structure analysis be trusted? Psychol Bull. 1992 Sep;112(2):351–62. doi: 10.1037/0033-2909.112.2.351 1454899
34. Peters G-J, Dima A, Plass AM, Crutzen R, Gibbons C, Doyle F. Measurement in health psychology: combining theory, qualitative, and quantitative methods to do it right: 6th Methods in Health Psychology Symposium. Eur Health Psychol. 2016;18(6):235–246.
35. Guillemin F, Bombardier C, Beaton D. Cross-cultural adaptation of health-related quality of life measures: literature review and proposed guidelines. J Clin Epidemiol. 1993 Dec;46(12):1417–32. doi: 10.1016/0895-4356(93)90142-n 8263569
36. Willis GB, Artino AR. What Do Our Respondents Think We’re Asking? Using Cognitive Interviewing to Improve Medical Education Surveys. J Grad Med Educ. 2013 Sep;5(3):353–6. doi: 10.4300/JGME-D-13-00154.1 24404294
37. van Kessel P, de Boer D, Hendriks M, Plass AM. Measuring patient outcomes in chronic heart failure: psychometric properties of the Care-Related Quality of Life survey for Chronic Heart Failure (CaReQoL CHF). BMC Health Serv Res. 2017 Aug 7;17(1):536. doi: 10.1186/s12913-017-2452-4 28784137
38. Ajzen I. Nature and operation of attitudes. Annu Rev Psychol. 2001;52:27–58. doi: 10.1146/annurev.psych.52.1.27 11148298
39. Hendriks AAJ, Smith SC, Chrysanthaki T, Black N. Reliability and validity of a self-administration version of DEMQOL-Proxy. Int J Geriatr Psychiatry. 2017;32(7):734–41. doi: 10.1002/gps.4515 27272129
40. Wijers IGM, Ayala A, Rodriguez-Blazquez C, Rodriguez-Laso A, Rodriguez-Rodriguez V, Forjaz MJ. Rasch Analysis and Construct Validity of the Disease Burden Morbidity Assessment in Older Adults. The Gerontologist. 2018 14;58(5):e302–10. doi: 10.1093/geront/gnx061 30219906
41. Plass AMC. Medical Care-Seeking and Self-Care Behaviour for Minor Illnesses. Amsterdam: VS&P; 2005.
42. Herrmann N, Rapoport MJ, Rajaram RD, Chan F, Kiss A, Ma AK, et al. Factor analysis of the Rivermead Post-Concussion Symptoms Questionnaire in mild-to-moderate traumatic brain injury patients. J Neuropsychiatry Clin Neurosci. 2009;21(2):181–8. doi: 10.1176/appi.neuropsych.21.2.181 19622689