Practice Analysis of Certified Practitioners in the Disciplines of Orthotics and Prosthetics

The practice analysis study of the O&P practitioner profession and the practitioner exam specifications involved a multi-method approach that included preliminary research, meetings with subject matter experts, and a survey. This section of the report describes the activities conducted for the practice analysis study. 

First, experts conducted research and identified the tasks, knowledge, and skills that they believed were important to the professional work done by O&P practitioners. Then, a survey was developed and disseminated to O&P practitioner professionals. The purpose of the survey was to obtain verification (or refutation) that the tasks, knowledge, and skills identified by the experts are important to the work of O&P practitioner professionals. Survey research functions as a “check and balance” on the judgments of the experts and reduces the likelihood that unimportant areas will be considered in the development of the test specifications. The use of a survey is also an efficient and cost-effective method of obtaining input from large numbers of experts and makes it possible for analysis of ratings by appropriate subgroups of respondents.

The survey results provide information to guide the development of test specifications and content-valid examinations. What matters most is that a certification examination covers the important tasks, knowledge, and skills needed to perform job activities. 

The steps of the practice analysis study are described in detail below.

1. Conduct of Role Delineation Research

ABC’s testing consultant, Prometric, in collaboration with ABC staff, conducted a review of available materials to develop draft task, knowledge, and skill statements. The primary source of information was the previous practice analysis study conducted in 2014. The tasks, knowledge and skills, practice areas, and additional questions from the previous survey were organized and prepared for review, revision, and validation by the members of the task force. Additional information from ABC regarding emerging trends and new areas of practice was also incorporated, as appropriate.

2. Development of the Survey

Conduct of the Practice Analysis Study Task Force Meeting

A task force meeting was held to develop the survey content, primarily consisting of the validated task, knowledge and skill statements, and practice areas and orthoses/prostheses. 

The task force was comprised of a group of ABC certified orthotists and prosthetists, who were selected to represent various levels of education, practice setting, job role, years of experience, and geographical diversity. In total, 21 volunteers comprised the task force. Table 43 summarizes their characteristics. 

Table 43

Survey Construction and Task Force Review

Upon the completion of the task force meeting, Prometric staff constructed the draft online survey. Each task force member then received a link to review the draft survey. The purpose of the review was to provide the task force with an opportunity to view their work and recommend any revisions.

Comments provided by the task force members were compiled by Prometric staff and reviewed with the task force members. Refinements, as recommended by consensus of the task force, were incorporated into the online survey in preparation for a pilot test. 

Survey Pilot Test 

The goal of the small-scale pilot test was for professionals in the field who had no previous involvement in the development of the survey to review the instrument and offer suggestions for improvement.  Pilot participants reviewed the survey for clarity of wording, ease of use, and comprehensiveness of content coverage. Comments were compiled by Prometric staff and reviewed with the task force members. The task force revised and finalized the survey based on the review of the pilot test comments.

Final Version of the Survey

The final version of the online survey consisted of eight sections: 

Section 1: Tasks
Section 2: Domains of Practice
Section 3: Knowledge and Skills
Section 4: Orthotic Practice Areas, Orthoses, and Additional Questions 
                                  OR 
                                  Prosthetic Practice Areas, Devices, and Additional Questions  
Section 5: Work and Patient Characteristics  
Section 6: Test Content Weightings
Section 7: Background & General Information
Section 8: Comments

In Survey Section 1: Tasks  
Job task statements were organized into the practice domains shown in Table 44.

Table 44

Participants rated each statement using the importance and frequency scales shown in Table 45.  Added emphasis was given to the Importance rating descriptions to clearly describe how respondents were to rate based on a newly certified practitioner. The Frequency rating related to the respondent’s own work experience. This was a change from the 2015 survey, which used the term Criticality instead of Importance, and where both questions referred to the respondent themselves, and not to a newly certified practitioner.

Table 45

In Survey Section 2: Domains of Practice
Participants indicated the percentage of time they spend performing tasks related to each domain during the most recent 12 months in which they provided direct patient care.

In Survey Section 3: Knowledge and Skills 
Survey participants rated the knowledge and skill statements using the importance scale shown in Table 46.

Table 46

In Survey Section 4: Orthotic OR  Prosthetic Practice Areas, Orthoses OR  Prostheses, and Additional Questions 
The survey used branching to direct participants to orthotics or prosthetics discipline-specific content. At the beginning of the survey, participants had been asked which credential they held. COs and CPs were automatically classified as orthotics and prosthetics, respectively. CPOs were given an additional question to select whether they primarily practice in orthotics or prosthetics and/or to indicate which discipline they prefer for the discipline-specific sections. Participants only responded to Section 4 if they had provided direct patient care in the past 12 months. The section consisted mainly of a series of questions related to percentage of time in practice areas, as well as other discipline-specific questions.

In Survey Section 5: Work and Patient Characteristics  
Participants gave additional percentage of time ratings in areas such as work setting or patient characteristics.

In Survey Section 6: Background & General Information  
Survey participants responded to general and background information questions about themselves and their professional activities.

3. Dissemination of the Survey

ABC disseminated the survey to all ABC certified orthotists, prosthetists and prosthetist/orthotists (n=5,504 on the date of dissemination). The survey remained open for five weeks. The invited survey participants received two reminder emails prior to the survey’s close. 

4. Analysis of the Survey Data

As previously noted, the purpose of the survey was to validate the tasks, knowledge, and skills that relatively large numbers of O&P practitioner professionals judged to be relevant (verified as important) to their work. This objective was accomplished through an analysis of the mean importance ratings for task, knowledge, and skill statements. The derivation of test specifications from those statements verified as important by the surveyed orthotists and prosthetists provides a substantial evidential basis for the content validity of credentialing examinations. For the purposes of this study, survey respondents were categorized as orthotics or prosthetics and analyzed in parallel. 

Based on information obtained from the survey, further data analyses by respondent subgroups (e.g., work setting, specialization, years of experience) are possible when sample size permits. A subgroup category was required to have at least 30 respondents to be included in the mean analyses. This was a necessary condition to ensure that the mean value based upon the sample of respondents is an accurate estimate of the corresponding population mean value.

The following quantitative data analyses were produced:

• Means, standard deviations, and frequency (percentage) distributions for task importance and content coverage ratings
• Means, standard deviations, and frequency (percentage) distributions for task frequency ratings
• Means, standard deviations, and frequency (percentage) distributions for knowledge and skill statements 
• Means and standard deviations for orthotic/prosthetic practice area ratings

5. Development of the Test Specifications 

The task force held a meeting to review the survey results and to develop test specification recommendations based on the practice analysis results. The goals of the meeting were to: 

1. Finalize the list of tasks, knowledge, and skills required upon entry to the profession

2. Recommend final exam content weightings for five exams: 

• Combined O&P Written Multiple-Choice Exam
• Orthotic Written Simulation Exam
• Prosthetic Written Simulation Exam
• Orthotic Clinical Patient Management Exam
• Prosthetic Clinical Patient Management Exam

The steps involved in the development of test specifications included the following:

• presentation of the practice analysis survey results
• finalization of the tasks, knowledge, and skills that are important for inclusion based on the survey results
• establishing the percentage test content weights for each area on each examination
• creating a linkage between the tasks and knowledge/skills

The main purpose of finalizing the task, knowledge, and skill statements and developing test content weights is to guide examination development activities. The test specifications for each of the orthotic and prosthetic practitioner exams are available on ABC’s website.