Avila BSW Program Assessment Plan and Continuous Improvement
The Avila University Social Work Program has made a commitment to achieving excellence in social work education. The faculty recognize that this value is demonstrated through use of effective assessment processes. In addition, the faculty know that such assessment is continuous, requiring skill, creativity, and dedication on the part of everyone in the department.
A comprehensive multimodal assessment plan has been developed, is being utilized, and includes activities and procedures for evaluating student attainment of ten core competencies, as demonstrated through achievement of forty-one associated practice behaviors. The Assessment Plan guides the measurement of the Program’s ability to advance graduates who have attained the requisite knowledge, skills, and values of professional social work practitioners. The ongoing collection of data and analysis of this data result in program viability and effectiveness by identifying potential areas for program growth and improvement, as well as affirming strengths that can be applied toward continuous improvement.
The Avila Social Work Program’s attainment of each Practice Behavior for 2015-2016 is assessed through: 1) Achievement Measure of Field Experience (AMFE); 2) Achievement Measure of Capstone (AMC).
PLAN TO ASSESS THE ATTAINMENT OF COMPETENCIES
The Program’s Assessment Plan is multimodal. Summative measures inform faculty of achievement in outcomes related to the Explicit Curriculum and the Implicit Curriculum. For the Explicit Curriculum, two primary measures (highlighted in blue in Chart 1) are used to evaluate student achievement of the forty-one practice behaviors and ten core competencies. The two primary measures used in the Assessment Plan are the Achievement Measure of Field Experience – Endpoint (AMFE – E) and the Achievement Measure of Capstone (AMC). The benchmark for the Program on both of these primary summative measures has been set at 80%. This figure represents the average percentage of students successfully meeting both of the measures for a practice behavior. This benchmark has been determined by all faculty.
Three supplemental measures of the Explicit Curriculum, Course-embedded Key Assignments, Senior Exit Survey, and the Alumni Survey, are also obtained for each practice behavior and core competency to provide several perspectives on student attainment of these desired outcomes. This design ultimately brings more clarity to conclusions drawn from data collected. These supplemental measures are for formative purposes only and are not a part of the summative assessment.
Achievement Measure of Field Experience –Endpoint (AMFE-E)
The AMFE-E is the first of the Social Work Program’s two primary, direct, summative measures of Explicit Curriculum. This tool measures student performance on each of the forty-one practice behaviors that operationalize the ten core competencies. Students take the pre-formatted tool and personalize it to their field agency and by developing at least two Learning Activities related to each practice behavior. These are written by the student in the form of SMART (Specific, Measureable, Attainable, Realistic, Timely) objectives. This process helps the student integrate and link knowledge, values, and skills necessary for attainment of practice behaviors and ultimately core competencies.
The first administration of the Achievement Measure of Field Experience happens at the Midpoint (December) of the field experience. This involves reviewing the student’s progress on their Learning Activities, as well as matching the descriptive standards on the measurement tool to the student’s performance during the field experience. The Endpoint (April) administration of this tool is completed by the Field Instructor, providing student, and Avila faculty with valuable feedback from a rater outside of the classroom, and working in a setting critical to student’s ultimate success. Aggregate data from the Endpoint collection using this tool informs student attainment of the Program’s core competencies.
As a formative measure, the AMFE – M results assist the student and the Field Instructor in noting the student’s own perceptions of their strengths and needs, how those compare to the Field Instructor’s ratings, and offer data to guide the necessary planning around development or modification of Learning Activities for the next semester in field. Students use the tool for self-evaluation at Midpoint and can thus compare and contrast their perception of their demonstrated achievement of knowledge, values, and skills with the evaluation of their Field Instructor.
Achievement Measure of Capstone (ACMR)
The Achievement Measure of Capstone (AMC) serves as the Program’s second primary measure. This measure utilizes a fixed measurement scale with corresponding levels of achievement and incorporates a standard description of exemplars for each level of attainment related to practice behaviors as follows:
- 0: Not Achieved - no concrete evidence of incorporation of the practice behavior in the assignment.
- 1: Minimally Achieved - incorporation of the practice behavior on a superficial level, for example student may allude to a concept with no corresponding information.
- 2: Moderately Achieved - incorporation of practice behavior in one instance with supporting material to explain reasoning.
- 3: Achieved - incorporation of practice behavior more than once with supporting material to explain reasoning in each instance.
- 4: Highly Achieved - incorporation of practice behavior multiple times, with supporting material, and a relation of the practice behavior to larger social work values/competencies.
Utilizing this scale as the guiding framework, the AMC is comprised of a set of rubrics that, taken in their entirety, measure all core competencies and practice behaviors. As every part of the Capstone Project measures a different subset of practice behaviors, each part of the Capstone Project has its own rubric. This equates to five rubrics total for the following parts of the assignments: Prospectus; Part 1: Problem Identification & Data Collection; Part 2: Micro/Macro Intervention; Part 3: Evaluation; and Part 4: Discussion. As an example, Part 1: Problem Identification and Data Collection, integrates 25 corresponding practice behaviors, and each practice behavior is evaluated utilizing the 0-4 scale and the standard description of exemplars outlined above. An external evaluator, who is a practicing social worker with a Master’s degree in Social Work, assesses student achievement on the Poster and Presentation at the Colloquium event, based on a rubric utilizing the 0-4 scale.
This measure also serves to provide formative data for students’ use in evaluation of their development. Each part of the Capstone assignment builds upon the next so students receive extensive descriptive comments on their level of achievement on each rubric with recommendations for editing moving forward. This feedback serves as the formative assessment for each part of the Capstone Project, and ultimately allows for the student to summarize the Project concisely and effectively at the Colloquium presentation.
Individual Student Ratings
During an academic year, each student is rated individually on their attainment of each Practice Behavior, using each of the two measures.
Aggregate of Student Ratings = Practice Behavior Attainment by two Measures
Next, for each of the two measures, for each Practice Behavior, all individual student ratings are averaged. This aggregation results in two rates (%) of student attainment for each distinct Practice Behavior.
Aggregate of two measure’s ratings = Program’s Attainment of Practice Behaviors
In order to obtain the Program’s final percentage of student attainment of a distinct Practice Behavior, the aggregate scores calculated for each of the two measures, are then averaged for each Practice Behavior. This computation provides the Program’s percentage of student attainment for each Practice Behavior, as indicated through the implementation of a multimodal assessment.
Aggregate of Practice Behavior Ratings = Program’s Attainment of Core Competencies
Finally, in order to calculate the Program’s student attainment rate (%) for each Core Competency, the Program’s student attainment rate for each Practice Behavior is used. All Practice Behaviors related to a distinct Core Competency provide a student attainment rate that is averaged with that of all of the Practice Behaviors clustered under that Competency. This calculation produces a final student attainment percentage for that Competency. This process is repeated for each Competency.
Benchmark for the Outcomes
BSW Program faculty have agreed to set the Benchmark level at an 80% or above achievement rate for all tools that measure the Explicit Curriculum and the Implicit Curriculum. Part of the consideration for these high levels of attainment is that students are required to have a 2.5 GPA and a minimum of “C” in all Social Work courses in the Avila Social Work Program in order to be retained in the Program. However, the faculty are not satisfied with graduating a lot of “C+” level Social Workers. Faculty generally agree that Program expectations are at the “B” to “A” level for the majority of our students. Faculty want students to advance to higher-level cognitive skills related to their Social Work values, knowledge, and skills. Faculty expect students to move beyond knowledge, and comprehension, to demonstrate analysis, synthesis, and evaluation at an appropriate undergraduate level. Avila has established a regional reputation that their Social Work Program graduates demonstrate a high level of performance in the requisite competencies for the field of Social Work. The established Benchmarks help students and faculty continue to strive to excel. In the spirit of respecting all individuals, Social Work faculty recognize that for some students a “C” may be their best level of achievement, and we embrace that awareness, while also encouraging that student to attain their highest potential possible during their time in the Social Work Program.
Data Analysis and Continuous Improvement as a Result of Assessment
Program attainment of the ≥ 80% benchmark on all core competencies and almost all practice behaviors demonstrates that overall, the Avila Social Work Program is achieving its goals and mission at a high level.
To obtain an understanding of specific strengths related to core competency attainment results were reviewed using the criterion that strength is demonstrated by final results ≥ 90% attainment for core competencies and ≥ 95% related to practice behaviors.
Recent Results of Assessment Activities
As the chart below indicates, based on evaluation of last year's assessment activities, Social Work graduates at Avila had attained all of the Core Competencies at the Benchmark rate by the time they completed their Social Work courses.
Publication of Assessment Results -
This Assessment Summary Report to all program constituencies: students, faculty, administration, Social Work alumni, Program advisory board, and others in the Kansas City regional community is updated annually on the Program web page.
Avila BSW Core Competency Achievement
Core Competency Achievement