ATSU Master of Science in Athletic Training program evaluation summary
Posted: June 10, 2016
ATSU Master of Science in Athletic Training program evaluation summary
During this past year, the Athletic Training program has been writing its self-study for CAATE accreditation. As part of the self-study process, a significant amount of time and attention has been dedicated to an evaluation of our assessment measures. This is important information to share with current students and alumni. A summary of our program evaluation is provided below. Students or alumni interested in more detailed information, or who have feedback regarding the outcomes assessment and reported outcomes, are encouraged to contact Dr. Tamara McLeod.
Athletic Training program purpose statement
The purpose of the master’s degree program in Athletic Training is to produce post-professional athletic trainers who practice with a high degree of professionalism, demonstrate advanced abilities to assess and diagnose athletic injuries and illnesses, and provide whole person, patient-centered care. Whole person, patient-centered care includes sound clinical decision-making, evidence-based practice, and clinical outcomes assessment.
Athletic Training program outcomes
Upon completion of the ATSU Post-Professional Graduate Athletic Training program, students will be able to achieve the following outcomes:
- Demonstrate safe and effective clinical decision-making in athletic training practice in a manner that integrates quality improvement, evidence-based practice, and patient-oriented outcomes.
- Demonstrate advanced knowledge and clinical practice skills in the examination, diagnosis, and management of sport related injuries and illnesses.
- Demonstrate advanced clinical practice, integrating effective communication, knowledge, skills, and clinical reasoning, professionalism, and self-reflection in daily practice.
- Demonstrate foundational knowledge of human anatomy.
- Demonstrate knowledge and skills for designing, conducting, analyzing, and disseminating athletic training research.
Outcomes supporting post-professional program
Program outcomes must increase students’ depth and breadth of understanding athletic training subject matter areas, skills, and Post- Professional Core Competencies, beyond the knowledge, skills, and abilities required of the professional preparation program.
The program objectives and the specific course level learning objectives were developed with specific attention to the fifth edition of the Athletic Training Educational Competencies with the goals of ensuring that, where appropriate, attainment of professional competence was assured and, in all instances, that specific learning in greater depth and breadth of that required for professional practice was required.
Evidence to support that students are learning athletic training content at a greater depth and breadth is provided by the 2015-2016 Student Performance Evaluation in which 100 percent of preceptors responding (20/20) agreed or strongly agreed that their students performed at a level above what would be expected of a newly certified, entry level athletic trainer. In addition, analysis of our AT Alumni Survey data from 2010-2015 demonstrated that the majority of students felt the program expanded their knowledge and skills beyond the entry-level:
- 95 percent agreed or strongly agreed the program expanded the depth and breadth of their entry-level applied, experiential, and propositional knowledge and skill.
- 97 percent of students agreed or strongly agreed the program facilitated their mastery of athletic training subject matter that was appropriate for a post-professional health professions master’s degree.
Analysis of our AT Employer Survey data from 2012-2014 demonstrated the majority of employers felt alumni had advanced knowledge and clinical practice skills beyond the entry-level:
- 92 percent agreed or strongly agreed alumni exhibited advanced knowledge and clinical practice skill in the area of physical examination and diagnosis.
- 85 percent agreed or strongly agreed alumni exhibited advanced knowledge and clinical practice skill in the area of orthopaedic rehabilitation.
- 100 percent agreed or strongly agreed alumni exhibited advanced knowledge and clinical practice skill in the area of evidence-based practice.
- 85 percent agreed or strongly agreed alumni exhibited advanced knowledge and clinical practice skill in the area of clinical outcome assessment, with 15 percent indicating they were unable to evaluate.
- 92 percent agreed or strongly agreed alumni exhibited advanced knowledge and clinical practice skill in the area of health informatics and technology, with 8 percent indicating they were unable to judge.
To date, we have only a small number of responses for Course Evaluations completed for courses which started since our major curricular revision was initiated at the start of the 2015 fall semester. Analysis of course evaluations from the current curriculum, suggest that course objectives related to the Post-Professional Core Competencies have extended the student’s knowledge and skills beyond their professional education.
However, for some course objectives, a higher percentage of students noted their knowledge and skills were not advanced, or only advanced minimally, beyond their professional education. Most notably, in HS 522: Research Design and Methods, 67 percent of students reported there was very little or no improvement in identifying and discussing ethical principles (beneficence, non-maleficience, autonomy, justice) and their use in guiding patient-centered care. In addition, 40 percent reported little or no improvement in their understanding of major ethical issues in clinical research. This may be due to some cursory research courses being required in professional programs that covers similar material.
In summary, analysis of our program evaluation data, including Course Evaluations, AT Alumni survey, AT Employer Survey, and Student Performance Evaluation provides sufficient evidence that the program’s outcomes increase students’ depth and breadth of understanding of athletic training subject matter areas, skills, and Post-Professional Core Competencies, beyond the knowledge, skills, and abilities required of the professional preparation program. Overall, we are very pleased with the assessments from students, alumni, and employers. While some outcomes were ranked lower, we feel this is due to some intentional redundancy in program content between what is offered within the program and what is offered through the core research courses offered by the School.
Evaluation of program outcomes
While we have not had a class go through the full two years with the new assessment plan, we relied on the analysis of data collected under our old assessment plan (through the class of 2015) when restructuring our curriculum.
As part of our Program Evaluation Plan we identified two areas to enhance student recruitment and retention. First, we have identified our overall tuition and lack of tuition waivers associated with graduate assistantship positions on campus as a barrier to recruiting students. As reported in our alumni survey (2010-2015), 26 percent of alumni were undecided, disagreed, or strongly disagreed that the program expenses (tuition, books, fees) were reasonable given the quality of education they received.
Second, we have set a goal to improve our two-year degree completion rate. Our current academic plan of study is for two years, however the ASHS Catalog allows for students to have a maximal degree completion timeline of five years. Since our last comprehensive accreditation visit, we have an average two-year degree completion rate of 56.3 percent, with 10.9 percent, 12.3 percent, and 4.1 percent completing in 3, 4, and 5 years respectively. The alumni survey results also suggest that 19 percent of alumni were undecided, disagreed, or strongly disagreed that sufficient time and opportunity was provided within the curriculum for students to complete a quality research experience including a hands-on experience with an established systematic method of inquiry.
Research Experiences: As a result of our assessments, changes were made in the structure of the Research Experiences when the curriculum was restructured starting with the class of 2016. The thesis courses were previously taught during the summer quarter, which were at the end of the student’s academic year. During the summer quarter, students had no other courses and were not required to be on campus for thesis work. In addition, the courses did not meet regularly and students were expected to complete all thesis requirements for each course, in collaboration with their advisor, over the course of the year. This may have led to procrastination and a lack of initiative to complete required assignments in a timely manner.
Since the implementation of the new curriculum and academic calendar, the Research Experiences begins earlier and at a time when demands of clinical practice are less intense. Students begin discussing possible projects with faculty earlier and have chosen a topic and advisor by the mid-point of their first block in the program. The Research Colloquia meetings also begin earlier, allowing students to get a head start on the literature review during their first year. The courses tied to the research aspects of the program have also changed.
The first year course, ATRN 5400: Research Practicum I is taught in the second block of the spring semester of the student’s first year. Dr. McLeod facilitates the course and has in-class meetings and checkpoint assignments due every other week. Time is also built into the course for oral proposals, giving students specific dates for which to propose.
Similarly, the second year course, ATRN 6400: Research Practicum II, is taught during the first block of the spring semester of the student’s second year. This course also meets in person and requires bi-weekly checkpoints to aid the student in completion of his/her manuscript and defense slides. By having this course the first block of the Spring Semester, it provides students time during the second block to complete any outstanding work and defend prior to the end of the academic year in May. While we only had one class complete ATRN 6400: Research Practicum II in the new format, we did have a 100 percent on-time thesis completion rate with the class of 2015. We will continue to evaluate the thesis completion rate, AT Alumni Survey responses, and student feedback with this new format and make additional changes as needed.
Advanced Clinical Practice Program: Beginning with the 2011-2012 academic year, the program began transitioning tracking of the Advanced Clinical Practice Program components to an electronic system (Typhon Group Allied Health Student Tracking [AHST]) from paper-based forms. During the 2011-2012 academic year, student clinical experience hours and self-reflection on select patient cases were tracked within the system, beginning in July. In the spring of 2011-2012, the Student Performance Evaluation was transitioned into the system. For the 2012-2013 academic year, the end of the Clinical Education Summary, Clinical Preceptor Evaluation, and Clinical Site Evaluation courses were all implemented within the AHST system. In 2013-2014, the Clinical Education Plan was moved into the electronic system, effectively eliminating paper tracking of advanced clinical practice components. With the revision of the curriculum for the 2014-2015 academic years, the Clinical Education Plan, Clinical Education Summary, and Student Performance Evaluations were revised to align with the new program outcomes and objectives.
Ongoing analysis of the Advanced Clinical Practice Program data has identified several trends to inform changes in the advanced clinical practice components of the program, especially during the 2014-2015 curricular revisions.
One specific trend was that students regularly reported high confidence in their knowledge and skills related to the Post-Professional Core Competencies, however they were not regularly integrating these skills into their daily clinical practice. From 2011-2016, students reflected on 8,549 patient encounters through the Patient Case Log form within the AHST system. In 81.7 percent of these encounters, students rated themselves as confident (2,935) or very confident (4,052) through their encounters and their skills.
Despite these high levels of confidence, students were not regularly reporting key professional behaviors such as evidence-based practice or clinical outcomes assessments. Students reported using evidence-based practice concepts in only 13.3 percent of patient encounters and collecting outcomes data in less than 1 percent of patient interactions. It should be noted that the Patient Case Logs are self-selected and self-reported by students, and as such, there is a possibility for bias in case selection. In addition, students may choose not to select all skills/procedures that were performed, resulting in incomplete data. As such, this information is used as a snapshot of Advanced Clinical Practice Program experiences, and in conjunction with other metrics of assessment and evaluation. These findings led to changes in clinical education activities, shifting the focus to improved integration of classroom knowledge and professional behaviors into clinical practice. Changes to the assessment measures used to capture information about the student’s Advanced Clinical Practice Program experiences were also identified. In 2015, quality improvement projects were implemented for each student, in order to provide the student with opportunities to engage in continuous quality improvement and to allow for more integration of the Post-Professional Core Competencies into their daily clinical practice each block.
Analysis of additional Advanced Clinical Practice Program data showed overall that students performed well in their clinical experiences, as evaluated by their clinical preceptors or supervisors. From spring 2012-spring 2015, the average score across all students was 3.7/4.0 or higher in the categories of Professional Behavior and Attitude, Safety, Interpersonal Relationships and Communication Skills, Documentation, Problem Solving, and Administrative and Management Skills. Additionally, students rated their clinical preceptors highly, with 92.6 percent of students identifying their clinical preceptor as an effective role model or mentor.
Global Program Assessment: It is important to note the AT Alumni Survey results from our previous curriculum (classes 2010-2015) demonstrate that the majority of students felt the program met the points of distinctiveness and program outcomes that were in place during their time as a student in the program. Specifically,
- 95 percent agreed or strongly agreed the program emphasized the development of advanced knowledge and clinical practice skills in the area of physical examination and diagnosis.
- 98 percent agreed or strongly agreed the program emphasized the development of advanced knowledge and clinical practice skills in the area of orthopaedic rehabilitation.
- 93 percent agreed or strongly agreed the program emphasized the development of advanced knowledge and clinical practice skills in the area of pediatric athletic training.
- 100 percent agreed or strongly agreed the program emphasized the provision of patient-centered whole person healthcare services emphasizing evidence-based practice.
- 100 percent agreed or strongly agreed the program emphasized the provision of patient-centered whole person healthcare services emphasizing clinical outcomes assessment.
- 98 percent agreed or strongly agreed the program emphasized the provision of patient-centered whole person healthcare services emphasizing healthcare informatics and technology.
- 95 percent agreed or strongly agreed the program knowledge of key healthcare dynamics in the United States.
- 100 percent agreed or strongly agreed the program emphasized the development of advanced knowledge and clinical practice skills in athletic training leadership, administration, and management.
- 98 percent agreed or strongly agreed the program emphasized the development of advanced knowledge and skills in the design, conduction, analysis, and dissemination of athletic training research.
ATSU Athletic Training Student Association update
The Athletic Training Student Association (ATSA) was once again very active during the 2015-2016 academic year. We gained 10 new members of the organization with the addition of the class of 2017. Among these new students, three served on the executive board and added greatly to the organization’s leadership. Throughout the year, the association maintained its service-oriented goals through participation in multiple events at ATSU and within the local community. Although a small organization, ATSA made a considerable impact on those they served.
To kick off the year, ATSA participated in a campus-wide interprofessional event to provide sports physicals for Special Olympic Arizona athletes during the National Day of Service. Our students contributed as clinicians, evaluating athletes during musculoskeletal screenings. Athletic training students paired up with first and second year physical therapy students providing guidance during musculoskeletal screenings, including muscular strength, flexibility, and neuromuscular reflexes. This was a positive experience for each student involved as they learned about each other’s programs and working with athletes with special needs.
ATSA continued building its relationship with Mesa Parks and Recreation through the Positive Play Project, providing coach and parent education on concussions, nutrition, hydration, and flexibility in the fall and spring seasons. This year, Mesa Parks also integrated two coach workshops where students provided practical training on proper warm up and stretching techniques to volunteer coaches. Students also engaged in community outreach and education during Celebrate Mesa, an event where they had an opportunity to interact with parents and young athletes.
In October, ATSA worked with ATSU’s Arizona School of Dentistry and Oral Health (ASDOH) to provide neuromuscular screenings during the annual Day for Special Smiles. This event offers free oral health evaluations and dental cleaning for patients with special circumstances. Day for Special Smiles was a great event for our students to interact with dental students and instructors, a group whom they rarely see on campus. Over the course of the day, 13 athletic training students assisted in the care of 78 patients, all of whom received over $15,000 in dental and medical care.
Continuing the long-held ATSA Christmas tradition, we were able to purchase gifts for eight children in the community through the Salvation Army’s Angel Tree program. A group of ATSA students met at a local Wal-Mart, selected children from the tree, and went shopping for gifts.
In February, Adam Annaccone, assistant athletic trainer for the Phoenix Suns, presented to students. The ATSU Sports Medicine Club invited Adam to speak about his role as an athletic trainer in professional sports, the sports medicine team at the Suns, and advice for students as they graduated and worked toward their professional goals. Athletic Training students were also involved in the Sports Medicine Club throughout the year, with presentations on functional movement screening, injury prevention, and sideline evaluations.
In March, the Athletic Training program hosted the annual ATSA High School Workshop for over 100 students from high school sports medicine programs in Arizona. The workshop included sessions on virtual anatomy, sideline concussion evaluation, rehabilitation and taping of the upper extremities, an ATSU professions student panel, and a “Quiz Bowl.” ATSA students developed teaching points and led each of the 40-minute sessions. They then created questions for the Quiz Bowl based off of the information in sessions.
The year ended with ATSA’s involvement in the campus-wide Free Community Health and Wellness Day. Athletic training students ran an obstacle course for children after their medical screenings. ATSA was responsible for providing education to parents and children on the importance of physical activity and nutrition. It was beneficial for students to work alongside other ATSU health professions students and to have a positive impact on the community.
Overall, this was a successful year for the Athletic Training Student Association. Engagement in interprofessional collaboration, leadership development, community education, and service drove the activities in which we participated. Thank you to the first years for contributing so much to the organization, and congratulations and good luck to the second years in future endeavors!