Authors: Rochelle E. Tractenberg aff001; Melody R. Wilkinson aff002; Amy W. Bull aff002; Tiffany P. Pellathy aff002; Joan B. Riley aff003 Authors place of work: Collaborative for Research on Outcomes and –Metrics, and Departments of Neurology, Biostatistics, Bioinformatics & Biomathematics, and Rehabilitation Medicine, Georgetown University, Washington, D.C., United States of America aff001; Department of Advanced Nursing Practice, School of Nursing & Health Studies, Georgetown University, Washington, D.C., United States of America aff002; Department of Nursing and Human Science, School of Nursing & Health Studies, and Center for New Designs in Learning and Scholarship, Georgetown University, Washington, D.C., United States of America aff003 Published in the journal: PLoS ONE 14(11) Category: Research Article doi: https://doi.org/10.1371/journal.pone.0224593
Advanced practice registered nursing (APRN) competencies exist, but there is no structure supporting the operationalization of the competencies by APRN educators. The development of a Mastery Rubric (MR) for APRNs provides a developmental trajectory that supports educational institutions, educators, students, and APRNs. A MR describes the explicit knowledge, skills, and abilities as performed by the individual moving from novice (student) through graduation and into the APRN career.
A curriculum development tool, the Mastery Rubric (MR), was created to structure the curriculum and career of the nurse practitioner (NP), the MR-NP. Cognitive task analysis (CTA) yielded the first of the three required elements for any MR: a list of knowledge, skills, and abilities (KSAs) to be established through the curriculum. The European guild structure and Bloom’s taxonomy of cognitive behaviors provided the second element of the MR, the specific developmental stages that are relevant for the curriculum. The Body of Work method of standard setting was used to create the third required element of the MR, performance level descriptors (PLDs) for each KSA at each of these stages. Although the CTA was informed by the competencies, it was still necessary to formally assess the alignment of competencies with the resulting KSAs; this was achieved via Degrees of Freedom Analysis (DoFA). Validity evidence was obtained from this Analysis and from the DoFA of the KSAs’ alignment with principles of andragogy, and with learning outcomes assessment criteria. These analyses are the first time the national competencies for the NP have been evaluated in this manner.
CTA of the 43 NP Competencies led to seven KSAs that support a developmental trajectory for instruction and documenting achievement towards independent performance on the competencies. The Competencies were objectively evaluable for the first time since their publication due to the psychometric validity attributes of the PLD-derived developmental trajectory. Three qualitatively distinct performance levels for the independent practitioner make the previously implicit developmental requirements of the competencies explicit for the first time.
The MR-NP provides the first articulated and observable developmental trajectory for the NP competencies, during and beyond the formal curriculum. A focus on psychometric validity was brought to bear on how learners would demonstrate their development, and ultimately their achievement, of the competencies. The MR-NP goes beyond the competencies with trajectories and PLDs that can engage both learner and instructor in this developmental process throughout the career.
Learning – Human learning – Allied health care professionals – Nurses – Health care policy – Careers – Nursing science – Metacognition
Academic progression, readiness for clinical practice, consistent assessment criteria, student empowerment of learning with transparent expectations, curriculum design, lifelong learning goals, and career development are all purposes that would be served by an Advanced Practice Registered Nurse (APRN) developmental trajectory of the knowledge, skills and abilities (KSA) of an APRN practice from education preparation to expert clinician. Competency based education is a starting point to achieve this outcome. NP competency-based education has been at the forefront of the NP educators’ agenda since 1980, with the publication of Guidelines for Family Nurse Practitioner Curricular Planning and the subsequent development of the National Organization for Nurse Practitioner Faculty (NONPF). In 2014, NONPF published the Nurse practitioner core competencies content: A delineation of suggested content specific to NP core competencies [1]. This collection of specific competencies are grouped into nine content areas (scientific foundations (4); leadership (7); quality (5); practice inquiry (6); technology and information literacy (7); policy (6); health delivery system (7) ethics (3); and independent practice (13)).
Competencies are articulated for many health and health science programs, including advanced practice nursing [1], medical informatics and bioinformatics [2, 3], and medicine [4]. The National Organization of Nurse Practitioner Faculty (NONPF) and the American Association of Colleges of Nursing (AACN) have led the development of competencies for advanced practice registered nursing (APRN) education.
Competencies may be intermediate milestones [5] or endpoints for graduates of programs, but howsoever they are placed, they encourage “outcomes-based” education [6–10]. They generally capture what programs within a discipline must do, teach, and provide for graduates, bringing meaning and comparability across such programs [11]. Competencies have been articulated for the ‘main’ purpose of programs, for example in physical therapy [12], pharmacy [13], and bioinformatics [3].
As the articulation of competencies is increasing across disciplines, so is the recognition that competencies can be difficult to fully integrate into a curriculum [14]. The challenges have been highlighted by the disciplines of public health [15], medical education [16] and bioinformatics [3]. Graduate nursing education also experiences challenges related to developing competency-based education. The American Association of Colleges of Nursing (AACN) echoes the importance of competency-based APRN education while also reiterating the lack of a formal framework to develop competency-based education [17]. APRN education models have not significantly changed in over four decades. National APRN education leaders currently advocate the development of competency-based models of education; however, there is no current structure for consistent implementation of competency-based education [18–20]. Creating a framework to guide competency-based education is dauntingly complex.
The national discussion about competency-based education extends across clinical programs. In a June 2017 special issue of Medical Teacher, a set of 11 articles provide overviews of the wide variety of challenges faced by medical education as faculty worldwide seek to promote competency-based medical education (CBME). In that special issue, Holmboe et al. [21] specify that the organization of “teaching and learning experiences… to facilitate an explicitly defined progression of ability in stages” (p.575) is a fundamental characteristic of CBME. The same conclusion–that a list of competencies requires a supporting progression—was independently drawn for bioinformatics competencies as well [16]. The challenges associated with implementing competency-based education are not limited to medical [21] and bioinformatics [16] programs. An explicit and publicly shared developmental trajectory for learners should be a fundamental characteristic of any higher education curriculum. Ideally, this progression should be consistent with the principles of adult learning (andragogy), and decisions about who has, and who has not yet achieved a given stage, should be informed by psychometric considerations of validity.
A Mastery Rubric (MR) is a curriculum development and evaluation tool that specifically provides such a trajectory. The first MR was created in 2005 and published in 2010 [22] to describe the clinical research training curriculum, including application and admissions criteria, for a two-year certificate program. Every MR requires three elements: first is a list of knowledge, skills, and abilities (KSAs1) that the curriculum exists to impart, all of which are concretely observable and evaluable even though they can be both taught and demonstrated at the target level of performance in a variety of ways [23, 24]. A MR also requires an explicit developmental trajectory, within which performance levels (stages) are identified with sufficient concreteness that the third MR feature, performance-level descriptions (PLDs) of each KSA at each level or stage, can be articulated–and recognized when met. PLDs represent standards against which any individual student’s work can be evaluated. The creation of a MR integrates the KSAs with the developmental trajectory across stages that are programmatically meaningful, and that promote sustainable learning–i.e., learning that can continue after formal instruction ends and that can be transferred to new contexts [25].
A curriculum based on a MR is intended to support instruction and assessment that generate actionable evidence [26, 27]. Actionable evidence is information that can be used to make evidence-informed decisions. This would be beneficial to the institution for curriculum [28]; the instructor(s) about their course, a class session, or their assessments [29]; and students about their learning strategies [30, 31].
One of the most important features of a MR is that it is shared with all stakeholders, including the learners—so they can be engaged as full partners in their education. A MR-based curriculum can thereby engage the learner in metacognitive development, so that gaps or weaknesses in their performance or understanding, rather than feelings of satisfaction or unease, identify areas for more learning or practice. Instructors and learners can then use the MR to document that these gaps/weaknesses have been addressed [24, 26, 31]. A MR can support both the formal learning within a curriculum and also supporting the learners and other decision makers throughout the career [32]. Whether described for a career [32] or a curriculum [33], using a MR can support active learner engagement and explicit development towards the learning objectives of the curriculum. That is, instructors can use the MR to design/evaluate or revise a curriculum while the learner or graduate can continue to document their achievements of post-graduate levels of performance on all KSAs, as appropriate.
This paper describes a MR created for the curriculum and career of the nurse practitioner (NP). Intended to be shared with learners and all faculty, the MR-NP was designed to facilitate both curriculum design and ongoing assessment by learners and instructors, “to ensure that trainees continue to progress” [34] towards the achievement of the NP competencies, within the curriculum and beyond it, into the career.
The MR-NP was developed in two phases, following formal methods. The first phase was a cognitive task analysis (CTA) to identify the constituent KSAs that underpin the core competencies articulated by the National Organization of Nurse Practitioner Faculties [1]. The CTA is explicated further in the (S1 Supplemental Materials). The second phase of developing the MR-NP was a facilitated standard-setting exercise with four expert nursing faculty, through which the PLDs were developed and refined. Both of these methods are described in greater detail below.
Once developed, validity evidence supporting the resulting MR-NP was then collected using Degrees of Freedom Analysis (DoFA) [35, 36]. The first DoFA assessed the alignment of the KSAs with the current NONPF competencies. The second DoFA validation explored the alignment of the MR-NP with the principles for learning outcomes articulation [27] for all participants (program, faculty, students). The final DoFA assessed alignment of a curriculum developed following the MR-NP with the principles of andragogy [37], and compared this alignment with that of a curriculum developed according to the original NONPF competencies without developmental considerations.
The five co-authors were engaged in the development of the MR-NP from October 2015 to April 2017 as part of a Community of Practice focused on the stewardly scholarship of teaching and learning. The first author is the developer of the Mastery Rubric construct and a cognitive scientist specializing in higher and post graduate education with experience supporting the development of KSAs for Mastery Rubrics. The other four authors are experienced NP educators with post-graduate specialty certifications. In addition to maintaining active clinical practices they have been full-time educators in nursing across multiple institutional settings for at least five years (maximum 34 years). One co-author serves as Senior Scholar at our university’s center for teaching excellence and also serves in a leadership position for education innovation. Two co-authors are Program Directors of specialty tracks of the NP Master’s degree program. Together we served as the subject matter experts (SMEs) for the CTA.
As noted, every MR requires a set of KSAs that the curriculum exists to transmit. An established method for understanding KSAs in any task or learning enterprise is a cognitive task analysis (CTA). There are five general steps in CTA: Collect preliminary knowledge/information; identify knowledge representations and organizations; elicit knowledge; analyze and verify data; and format results [38]. In the NP context, the first three steps of the CTA were already completed and published as the NONPF Competencies. These represent a rich dataset from which concepts, processes and principles were extracted (see S1 Supplemental Materials).
These first three CTA steps led to a first draft of KSAs based on the NONPF competencies, which used Bloom’s taxonomy to extract the minimum cognitive activities needed for each competency [40]. Analysis and verification of the KSAs was accomplished through face-to-face and online meetings to discuss the CTA results until consensus was reached. Specifically, we met bi-weekly to evaluate and work with the derived KSAs, using the NONPF competencies and the SMEs’ individual experiences as nurse practitioners and faculty. We also planned that, once the MR-NP was drafted (i.e., results formatted, CTA step 5), we would further verify these results by alignment (DoFA) of the final version of our KSAs with the NONPF competencies to ensure that every KSA was required for multiple competencies, and any KSA that did not support at least two competencies would then be evaluated for its relevance. The CTA was not going to yield KSAs that were directly representative of any competency, so assessing how the CTA results (i.e., the KSAs) aligned with those competencies was an important validation step for the KSAs.
As described earlier, every MR requires a set of at least two developmental stages to be articulated for evaluable growth in each KSA, as well as performance level descriptors (PLDs) of each KSA at each of these stages. Together, the establishment of stages and the drafting of PLDs were accomplished through the standard setting exercise. The stages for the MR-NP were initially loosely based on decision-making steps in the NP program (admission to the program; admission to clinical preceptored experiences; graduation from the program; and two additional professional performance levels beyond graduation; see [24] for similar career-spanning stages in medical education).
The stages of most MRs [24] are modeled after the European guild structure: Novice, Apprentice, Journeyman [39]. This structure is chosen purposefully for MR staging because it captures a progressive sequence within which students can see themselves changing from less independent to more independent within the curriculum. Moreover, in this structure, the Apprentice is recognized as actively engaged in learning the craft or practice, while the Journeyman is the independent practitioner. These characteristics also signal to all instructors in a curriculum that students must evolve to progress–student evolution must be supported and the purpose of the curriculum is to facilitate these changes in observable and evaluable ways. Finally, recognizing that the end of a curriculum cannot represent the end of learning for the nurse practitioner, we supplemented the decision-making steps in the typical NP program with concrete career stages that represent distinct growth in responsibilities as well as abilities after graduation.
The final feature of a MR is drafting the PLDs. We iteratively articulated PLDs for each KSA across the decision points in the NP curriculum, and into the NP career, using a Body of Work approach [40]. Specifically, the first pass at drafting a PLD was "range finding" for performance of the KSA at a given stage, and the second pass was "pinpointing" [40], as described below. PLD drafting used Bloom’s taxonomy refined by appealing to the elements of psychometric validity in assessment, as outlined by Messick (1994) [41]:
What is/are the knowledge, skills, and abilities (KSAs) that students should possess at the end of the curriculum?
What actions/behaviors by the students will reveal these KSAs?
What tasks will elicit these specific actions or behaviors?
The integration of Bloom’s taxonomy and the Messick criteria in our standard-setting exercise facilitated the inclusion of concrete and observable behaviors in the PLDs that can be developed sequentially–and reinforced iteratively for deeper and sustainable learning over time. In our implementation of Messick questions 2 and 3, we considered what Bloom’s level the minimum specific actions/behaviors demonstrating that the KSA had been learned would need to be demonstrated for the learner to qualify for a given stage (e.g., “Apprentice”); this is the range-finding. The pin-pointing is accomplished by refining PLDs (see S1 Supplemental Materials) so that each KSA can be recognized at each stage for any sample of student work.
This approach ensures that a curriculum that is developed, or revised, to follow this developmental path for each KSA will likely support evaluable learning goals, and their achievement. Competencies, in general, are multi-dimensional and complex behaviors, whereas learning goals should be simple and not compound, among other things. This may be one reason why there has been difficulty in creating competencies-based educational experiences [3, 14–16]. The MR approach to curriculum development also increases the likelihood of actionable evidence [27, 28] for the institution as well as for instructors/course directors and students, because a MR is always intended to be made public and shared [36]. These features also add evaluability to the PLDs themselves, such that a curriculum or continuing education experience that follows the MR-NP (or one that promotes students following it) can be evaluated in terms of its achievement of intended results [6].
Degrees of Freedom Analysis (DoFA) [35] is a qualitative method for assessing the alignment of data (observations) with predictions from theory or decisions [36, 42]. The recent modifications for using DoFA in educational contexts are specifically intended to summarize evidence surrounding decisions about teaching, learning, and assessment [36]. Validity evidence for the MR-NP was compiled from three DoFA matrices, assembling the evidence that it is:
sufficiently aligned with the NONPF competencies to conclude that a curriculum based on the MR-NP could support the achievement of these competencies;
consistent with principles of learning outcomes articulation and assessment, supporting a conclusion that a curriculum based on the MR-NP could promote actionable and evaluable learning outcomes, and
well-aligned with principles of andragogy, supporting a conclusion that a curriculum based on the MR-NP would also be so aligned.
Andragogic principles and National Institute for Learning Outcomes Assessment (NILOA) [27] characteristics are important, although underutilized, in higher, graduate, and professional education. The KSAs in the MR-NP, and their developmental trajectories, must support achievement of the consensus-based competencies if anyone would choose the MR-NP for curricular support. This validity evidence supports the decision to use the MR-NP in NP curriculum development.
The results of the Cognitive Task Analysis and standard setting exercise (i.e., the MR-NP) appear in Table 1. Table 2 presents the DoFA testing the alignment of the KSAs with the NONPF Competencies. Table 3 presents the DoFA alignment of the MR-NP with the NILOA criteria for actionable learning outcomes [27] while Table 4 outlines the alignment of the MR-NP with principles of andragogy [37] comparing this alignment with that of the competencies on which the MR-NP was based.
The cognitive task analysis (see S1 Supplemental Materials) initially identified five KSAs as underpinning the NONPF competencies: prerequisite knowledge; communication skills; reflection and metacognition; data and evidence skills; and clinical practice. Two additional KSAs were identified and included/retained: One was needed to accommodate NONPF competencies that fell between “data and evidence skills” and “clinical practice”: this became “translation for evidence-based practice”. The other was needed to capture engagement with policy and advocacy:“policy/advocacy.” As the developmental paths through the performance-level descriptors for these KSAs were developed, both of them retained their differentiations from the other KSAs. All seven KSAs are presented in the MR-NP shown in Table 1.
The Body of Work approach [40] to standard setting, i.e., writing the PLDs that capture standards of performance on each KSA at each stage, was used to draft and revise descriptions of how a “minimally competent individual” [43] would carry out the KSAs at each stage to demonstrate that they were capable of performing at that level.
As can be seen in Table 1, the MR-NP represents the stages in the developmental trajectory of the NP from admission to the program (“Novice”) through their qualification for clinical precepted experiences (“Apprentice”), graduation (“Journeyman 1”) and across their career trajectory (“Journeyman 1” through “Journeyman 3”). Development is implicit, but not articulated, in the NONPF Competencies.
The NP curriculum decision points were used in the first drafts of PLDs, and the final PLDs were revised specifically to characterize the individual at these stages: accepted into a formal training program (“Novice”), the individual who can document their preparation for preceptored clinical experiences (“Apprentice”) and the individual who is newly independent (“Journeyman”). Evidence for admission into an NP program in the United States (based on the requirements for Bachelor’s of Science for Nursing or BSN [44]), and preparedness for clinical preceptorship are consistent with novice and apprentice level performance respectively; graduation from the NP program can be considered to represent an individual’s qualification for independent practice. The complex nature of professional development for the NP beyond the end of formal education is recognized by the inclusion of three distinct levels of “Journeyman”; these were dictated in part by the opportunities and requirements that the NONPF competencies themselves suggest.
The first row of the MR-NP provides a general description of overall performance of the minimally-qualified individual at each level [45]. Once the KSAs were extracted and PLDs were drafted for the Novice level, we determined that the PLDs were consistent with our program’s admission process, providing content validity for that set of PLDs. Three distinct Journeyman stages emerged: the recent graduate is described by Journeyman-1, the individual who has NP experience that must accrue after graduation is at the Journeyman-2 level, and Journeyman-3 describes the recognized NP leader with NP practice expertise. All Journeyman-level individuals are qualified for independent practice; the fact that the standard setting led to three differentiable levels of independence is a function of the competencies themselves. As can be seen in Table 1, the PLDs of all KSAs except “prerequisite knowledge” and “clinical practice” are the same for Journeyman 1–2. Journeyman 3 performance level descriptors characterize the practitioner with sufficient experience to be considered a “recognized leader,” qualitatively different from Journeyman 1 and 2. These levels of performance differ as experience changes to expertise–making implicit developmental requirements of the NONPF competencies, beyond the formal curriculum and throughout a career, explicit for the first time.
Validity evidence for the MR-NP was obtained by appeal to extant criteria. First, Table 2 explores alignment of the KSAs with the individual NONPF competencies.
As Table 2 reflects, most of the 49 competencies articulated in the NONPF statement [1] are supported by at least one KSA. However, not all of these competencies are specifically dependent on the KSAs outlined in the MR-NP. We found that six of 49 competencies could not be aligned with any of the KSAs in the MR-NP; the remaining 87% of the NONPF competencies were aligned with at least two KSAs. Failures in alignment between the KSAs and the competencies were identified independently by different co-authors. Two characteristics were determined to have prevented alignment of KSAs with these six competencies. Either the competency was determined not to have been described concretely enough to identify a clear KSA or instructional approach, or else the competency was evaluable, and did not require any of the KSAs in order to be achieved. These are identified as such within Table 2.
Further validation of the MR-NP was sought by appeal to the 2016 recommendations for actionable learning outcomes published by the National Institute for Learning Outcomes Assessment (NILOA, [27]). NILOA was established to encourage educators nationwide to develop evaluable and robust learning outcomes to improve student learning and higher education generally. Their 2016 policy statement [27] includes five important attributes of meaningful learning outcomes. Table 3 presents the alignment of the MR-NP features with the predictions derived from these five NILOA principles for documenting learning outcomes, plus four additional characteristics of robust learning outcomes highlighted in Tractenberg, et al. (2017) [25].
Table 3 shows how the MR-NP can be used to ensure that learning outcomes for courses and the curriculum are as explicit for students as they are for faculty and the program or institution. A curriculum that is created or revised using the MR-NP will exhibit all five of the NILOA learning outcome characteristics, generating actionable evidence for students as well as for the program and faculty engaged in the curriculum. However, a curriculum that is based on just the NONPF competencies will not generate such evidence. This is a function of the implicit nature of any developmental trajectory in the competencies; the MR-NP makes development explicit, and the PLDs were formulated specifically to conform to psychometric validity [34, 41]. The competencies offer only summative opportunities for assessment while the MR-NP supports formative assessment as well.
Finally, a MR is intended to promote curriculum development, evaluation, or revision that is consistent with principles of andragogy [37]. Table 4 is a descriptive analysis that presents how the MR-NP embodies this alignment.
Table 4 shows concrete differences in alignment with andragogical principles that would arise from a curriculum that is developed to feature the NONPF competencies [1] and one that was developed (or revised) using the MR-NP. This is not an empirical analysis, but a conceptual one. While both types of curricula could lead students to achieve the NONPF competencies, the curriculum developed using the MR-NP would also be consistent with the principles of andragogy, thereby engaging learners as partners in their own development and promoting reflection and metacognition to a greater extent [25].
A new MR was created using published national competencies for nurse practitioner (NP) education, together with a formal and iterative standard setting exercise. Seven key areas of knowledge, skills, and abilities (KSAs) were derived so that an NP curriculum can go beyond simply aligning with, or including, the competencies. The MR-NP (Table 1) can be used to develop and evaluate courses and curricula, as well as teaching, learning, and practice. It provides a roadmap to addressing many challenges in NP education: supporting the explicit, but flexible, articulation of valid learning outcomes; promoting growth in students along a newly-specified developmental trajectory; and leading to the achievement of the target competencies—in the curriculum and beyond, into the career of the NP. The structure can be used to create consistency across programs, faculty, and students, enabling specificity and measurability in assessment. By making expectations transparent to all stakeholders, the MR-NP empowers self-directed growth.
The MR-NP has face validity that derives partly from its support of/alignment with the national competencies that are at its core (Table 2), with additional validity accruing from the application of cognitive task analysis emphasizing Bloom’s taxonomy [46] with formal and iterative standard-setting. The infusion of psychometric validity in the PLDs yields a usable and evaluable curriculum description (Table 1). The DoFA analysis on the consistency of the MR-NP, with principles of learning outcomes assessment (Table 3), highlights how existing NP curricula can be revised to generate evidence about learning for both the student and the instructor/institution; these revisions are not clear with the competencies but are accessible with the MR-NP. This DoFA replicates the results from two other MRs for Statistical Literacy [26] and for Ethical Reasoning [32], suggesting that a curriculum that employs the MR approach can promote this actionable evidence. Finally, a curriculum that utilizes the MR-NP is aligned with principles of andragogy (Table 4) in ways that a curriculum that utilizes just the competencies would not be, even if the competencies are shared with learners as fully as the MR-NP is intended.
The creation of the MR-NP is the first step in our educational research program, and as such, we have not had the opportunity to empirically study curricula that do and do not utilize it. The methodology and formal incorporation of psychometric validity, learning outcomes assessment, andragogy principles, and cognitive complexity will hopefully encourage NP curriculum developers to adopt the MR-NP and report on their results.
A MR can engage the learners in the structure and achievement of learning objectives throughout their career [32]. While the competencies represent targeted end states, the MR-NP promotes ongoing engagement with its core KSAs past graduation and well into NP professional practice and ultimately, leadership roles. By developing concrete, measurable objectives that are supportive of the NONPF competencies—across an explicit educational trajectory—nurse faculty and students can document individual accomplishment of competencies as they are developed over time. Students who are not achieving competencies as expected can/may identify the need and opportunity for remedial work focused on the area of deficiency rather than repeating an entire course or an additional prescribed number of clinical hours.
Faculty development and advancement, curriculum evaluation, and classroom innovation can each/all be facilitated when curricular structure is used to align and sequence courses and learning objectives. When formal structure explicitly underpins all components of a curriculum like the MR-NP suggests, both teaching and learning can be made more evaluable. This evaluability can strengthen the curriculum generally, while simultaneously augmenting the likelihood of the success of all students. Anecdotally, all of these claims were supported when the practitioner co-authors utilized the MR-NP in their teaching, mentoring, and student assessment. Because these experiences were not formal or systematic, they are offered as preliminary evidence that encourage future formal studies regarding the implementation and usability of the MR-NP.
The MR-NP flexibly represents how the NONPF competencies, including the most cognitively-complex ones, can be achieved within the NP curriculum by integrating a developmental trajectory that increases the cognitive complexity of instruction and work products across the curriculum and NP career. This MR-NP provides a straightforward, useful tool for instructors to use when revising their objectives and syllabi. The usability of the MR-NP increases the potential for buy-in and uptake within APRN programs. Buy-in, commitment of time and resources, and a formal evaluation plan are all required if the integration of competencies is to be done for all courses in the curriculum [47].
The MR-NP provides clear descriptors of when an APRN student is competent to transition into supervised clinical practice and provided a framework for demonstrating that competencies for independent clinical practice have been met. Without a developmental trajectory, the competencies alone might not support the “transition to practice” [48]; however, the trajectory outlined by the MR-NP might. Our future work will focus on examining the uptake and buy-in to this particular MR-NP at our institution beyond our research group, and the generation of actionable evidence that supports the accreditation claim that the target NONPF competencies are not simply aspirational, but descriptive [49]. The support of the MR-NP of transition to practice is another avenue for research.
In conclusion, these results support the use of an explicit developmental trajectory, like the MR-NP, to promote learning and assessment to ensure that competencies are both achieved by students and explicitly supported throughout the curriculum and throughout the transition to practice.
S1 Supplemental Materials [docx]
1. Thomas, A., Crabtree, M. K., Delaney, K., Dumas, M. A., Kleinpell, R., Marfell, J, et al. Nurse practitioner core competencies content: A delineation of suggested content specific to NP core competencies. 2014. http://c.ymcdn.com/sites/nonpf.site-ym.com/resource/resmgr/Competencies/NPCoreCompsContentFinalNov20.pdf
2. Kulikowski CA, Shortliffe EH, Currie LM, Elkin PL, Hunter LE, Johnson TR, et al. AMIA Board white paper: definition of biomedical informatics and specification of core competencies for graduate education in the discipline. Journal of the American Medical Informatics Association, 2012. 19(6): p. 931–938. doi: 10.1136/amiajnl-2012-001053 22683918
3. Welch L, Lewitter F, Schwartz R, Brooksbank C, Radivojac P, Gaeta B, et al., Bioinformatics curriculum guidelines: toward a definition of core competencies. PLOS computational biology, 2014. 10(3): p. e1003496. doi: 10.1371/journal.pcbi.1003496 24603430
4. Holmboe ES, Edgar L, Hamstra S, The milestones guidebook. Chicago, IL: Accreditation Council for Graduate Medical Education, 2016.
5. Englander R, Frank JR, Carraccio C, Sherbino J, Ross S, Snell L, & on behalf of the ICBME Collaborators. Toward a shared language for competency-based medical education. Medical teacher, 2017. 39(6): p. 582–587. doi: 10.1080/0142159X.2017.1315066 28598739
6. Bresciani M.J., Outcomes-based academic and co-curricular program review: A compilation of institutional good practices. 2006. Sterling, VA: Stylus.
7. Tekian A, Hodges BD, Roberts TE, Schuwirth L, Norcini J., Assessing competencies using milestones along the way. Medical teacher, 2015. 37(4): p. 399–402. doi: 10.3109/0142159X.2014.993954 25523010
8. Wagner S.J. and Reeves S., Milestones and entrustable professional activities: the key to practically translating competencies for interprofessional education? Journal of interprofessional care, 2015. 29(5): p. 507–508. doi: 10.3109/13561820.2014.1003636 26062110
9. Scott Tilley DD, Allen P, Collins C, Bridges RA, Francis P, Green A., Promoting clinical competence: Using scaffolded instruction for practice-based learning. Journal of Professional Nursing, 2007. 23(5): p. 285–289. doi: 10.1016/j.profnurs.2007.01.013 17903787
10. Carter KF, Kaiser KL, O’Hare PA, Callister LC. Use of PHN competencies and ACHNE essentials to develop teaching–learning strategies for generalist C/PHN curricula. Public Health Nursing, 2006. 23(2): p. 146–160. doi: 10.1111/j.1525-1446.2006.230206.x 16684189
11. Ten Cate O, Billett S., Competency‐based medical education: origins, perspectives and potentialities. Medical Education, 2014. 48(3): p. 325–332. doi: 10.1111/medu.12355 24528467
12. American Physical Therapy Association, Competencies of the Transition DPT Graduate. 2015. http://www.apta.org/PostprofessionalDegree/GraduateCompetencies/
13. Burke JM, Miller WA, Spencer AP, Crank CW, Adkins L, Bertch KE, et al., Clinical pharmacist competencies: American College of Clinical Pharmacy. Pharmacotherapy: The Journal of Human Pharmacology and Drug Therapy, 2008. 28(6): p. 806–815.
14. Fleming ML, Parker E, Gould T, Service M., Educating the public health workforce: issues and challenges. Australia and New Zealand Health Policy, 2009. 6 : 8. doi: 10.1186/1743-8462-6-8 19358714
15. Caverzagie KJ, Nousiainen MT, Ferguson PC, Ten Cate O, Ross S, Harris KA, et al., Overarching challenges to the implementation of competency-based medical education. Medical teacher, 2017. 39(6): p. 588–593. doi: 10.1080/0142159X.2017.1315075 28598747
16. Welch L, Brooksbank C, Schwartz R, Morgan SL, Gaeta B, Kilpatrick AM, et al., Applying, evaluating and refining bioinformatics core competencies (an update from the curriculum task force of ISCB’s Education Committee). PLoS computational biology, 2016. 12(5):e1004943. doi: 10.1371/journal.pcbi.1004943 27175996
17. American Association of Colleges of Nursing, Re-envisioning the clinical education of advanced practice registered nurses (White paper). 2015.
18. LeFlore J.L. and Thomas P.E., Educational changes to support advanced practice nursing education. The Journal of perinatal & neonatal nursing, 2016. 30(3): p. 187.
19. Giddens J.F., Lauzon-Clabo L, Morton PG, Jeffries P, McQuade-Jones B, Ryan S., Re-envisioning clinical education for nurse practitioner programs: themes from a national leaders’ dialogue. Journal of Professional Nursing, 2014. 30(3): p. 273–278. doi: 10.1016/j.profnurs.2014.03.002 24939338
20. Sroczynski M, Dunphy LM., Primary care nurse practitioner clinical education: Challenges and opportunities. Nursing Clinics, 2012. 47(4): p. 463–479. doi: 10.1016/j.cnur.2012.08.001 23137599
21. Holmboe ES, Sherbino J, Englander R, Snell L, Frank JR for the ICBME Collaborators A call to action: the controversy of and rationale for competency-based medical education. Medical Teacher, 2017. 39(6): p. 574–81. doi: 10.1080/0142159X.2017.1315067 28598742
22. Tractenberg RE, Umans J, McCarter RJ, A Mastery Rubric: Guiding curriculum design, admissions and development of course objectives. Assessment & Evaluation in Higher Education, 2010. 35(1): p. 15–32.
23. Tractenberg, RE. Developing a curriculum for research in pathology residency: A Pathology Research Mastery Rubric. Poster presented at the Northeast Group on Educational Affairs (NEGEA) Annual Meeting, Washington, DC, USA. 2011.
24. Tractenberg, RE., The Mastery Rubric: A tool for curriculum development and evaluation in higher, graduate/post-graduate, and professional education. Open Archive of the Social Sciences (SocArXiv), 2017.
25. Tractenberg RE, FitzGerald KT, and Collmann J, Evidence of sustainable learning from the Mastery Rubric for ethical reasoning. Education Sciences, 2017. 7(1):2.
26. Tractenberg RE, How the Mastery Rubric for Statistical Literacy can generate actionable evidence about statistical and quantitative learning outcomes. Education Sciences, 2017. 7(1):3.
27. National Institute for Learning Outcomes Assessment, Higher Education Quality: Why Documenting Learning Matters. 2016. Urbana, IL: University of Illinois and Indiana University
28. Kuh G. D., Ikenberry S. O., Jankowski N., Cain T. R., Ewell P. T., Hutchings P., & Kinzie J. Using evidence of student learning to improve higher education. 2015. San Francisco, CA: Jossey-Bass.
29. Hutchings, P., Kinzie J, and Kuh G.D., Evidence of student learning. In GD Kuh, SO Ikenberry, & NA Jankowski (Eds.). Using evidence of student learning to improve higher education, 2015: p. 17–50.
30. Hutchings P., Aligning Educational Outcomes and Practices. Occasional Paper# 26. Urbana, IL: University of Illinois and National Institute for Learning Outcomes Assessment, 2016.
31. Ambrose SA, DiPietro MW, Lovett MC, Norman MK, How learning works: Seven research-based principles for smart teaching. 2010. San Francisco, CA: John Wiley & Sons.
32. Tractenberg RE and FitzGerald KT, A Mastery Rubric for the design and evaluation of an institutional curriculum in the responsible conduct of research. Assessment & Evaluation in Higher Education 2012. 37(8): p. 1003–1021.
33. Tractenberg RE, Gushta MM, Weinfeld JM, The Mastery Rubric for evidence-based medicine: Institutional validation via multidimensional scaling. Teaching and learning in medicine. 2016. 28(2): p. 152–165 doi: 10.1080/10401334.2016.1146599 27064718
34. Lockyer J, Carraccio C, Chan MK, Hart D, Smee S, Touchie C, et al., Core principles of assessment in competency-based medical education. Medical teacher, 2017. 39(6): p. 609–616. doi: 10.1080/0142159X.2017.1315082 28598746
35. Campbell D.T., “Degrees of freedom” and the case study. Comparative political studies, 1975. 8(2): p. 178–193.
36. Tractenberg RE, Degrees of freedom analysis in educational research and decision-making: leveraging qualitative data to promote excellence in bioinformatics training and education. Briefings in Bioinformatics, 2019. 20(2): p. 416–425. doi: 10.1093/bib/bbx106 30908585
37. Knowles MS, Holton III, E F, Swanson RA, The adult learner 6th ed. 2005. Burlington MA: Elsevier.
38. Clark RE, Feldon DF, van Merriënboer J, Yates KA, Early S., Cognitive Task Analysis in Michael Spector J., David Merrill M., Elen Jan, Bishop M. J. Eds. Handbook of research on educational communications and technology, 3rd ed. 2008. Mahwah, NH: Lawrence Earlbaum Associates. 577–593.
39. Olgivie S., The economics of guilds. Journal of Economic Perspectives, 2014. 28(4): p. 169–92.
40. Kingston NM, Tiemann GC., Setting Performance Standards on Complex Assessments: The Body of Work Method, in Setting Performance Standards: Foundations, Methods, and Innovations, Cizek GJ, Editor. 2012. New York, NY: Routledge. Pp 201–223.
41. Messick S., The Interplay of Evidence and Consequences in the Validation of Performance Assessments. Educational Researcher, 1994. 23(2): p. 13–23.
42. Woodside A., Case study research: Theory, methods and practice. 2010. Bangles, UK: Emerald Group Publishing.
43. Plake BS, Cizek GJ. Variations on a Theme: The modified Angoff, Extended Angoff, and Yes/No standard setting methods., in Setting performance standards. Foundations, methods, and innovations, Cizek GJ, Editor. 2012. New York, NY: Routledge. Pp 181–199.
44. American Association of Colleges of Nursing, The Essentials of Baccalaureate Education for Professional Nursing Practice. 2008.
45. Cizek GJ, An introduction to contemporary standard setting: concepts, characteristics, and contexts, in Setting Performance Standards: Foundations, Methods, and Innovations, Cizek GJ, Ed. 2012. New York, NY: Routledge. pp. 3–14.
46. Bloom BS (Ed.), with Engelhart MD, Furst EJ, Hill WH & Krathwohl DR. Taxonomy of educational objectives: Handbook I: Cognitive domain. 1956. New York, NY: David McKay.
47. Diamond RM., Designing and assessing courses and curricula: A practical guide. 3rd ed. 2008. San Francisco, CA: Jossey Bass.
48. MacKay M, Glynn D, McVey C, Rissmiller P., Nurse practitioner residency programs and transition to practice. Nursing Forum, 2018. 53(2): p. 156–160. doi: 10.1111/nuf.12237 29076545
49. Egan KL, Schneider C, & Ferrara S, Performance level descriptors: History, practice, and a proposed framework., in Setting Performance Standards: Foundations, Methods, and Innovations, Cizek GJ, Editor. 2012. New York, NY: Routledge. pp. 79–106.
Zvyšte si kvalifikaci online z pohodlí domova
Nemáte účet? Registrujte se
Zadejte e-mailovou adresu, se kterou jste vytvářel(a) účet, budou Vám na ni zaslány informace k nastavení nového hesla.
