Avila BSW Program Assessment Plan and Continuous Improvement
DEPARTMENT OF SOCIAL WORK ASSESSMENT REPORT
The Avila University Social Work Program has made a commitment to achieving excellence in social work education. The faculty recognize that this value is demonstrated through use of effective assessment processes. In addition, the faculty know that such assessment is continuous, requiring skill, creativity, and dedication on the part of everyone in the department.
A comprehensive multimodal assessment plan has been developed, is being utilized, and includes activities and procedures for evaluating student attainment of ten core competencies, as demonstrated through achievement of forty-one associated behaviors. The Assessment Plan guides the measurement of the Program’s ability to advance graduates who have attained the requisite knowledge, skills, and values of professional social work practitioners. The ongoing collection of data and analysis of this data result in program viability and effectiveness by identifying potential areas for program growth and improvement, as well as affirming strengths that can be applied toward continuous improvement.
The Avila Social Work Program’s attainment of each Behavior for 2017-2018 is assessed through: 1) Achievement Measure of Field Experience (AMFE); 2) Achievement Measure of Capstone (AMC).
PLAN TO ASSESS THE ATTAINMENT OF COMPETENCIES
The Program’s Assessment Plan is multimodal. Summative measures inform faculty of achievement in outcomes related to the Explicit Curriculum and the Implicit Curriculum. For the Explicit Curriculum, two primary measures (highlighted in blue in Chart 1) are used to evaluate student achievement of the behaviors and nine core competencies. The two primary measures used in the Assessment Plan are the Achievement Measure of Field Experience – Endpoint (AMFE – E) and the Achievement Measure of Capstone (AMC). The benchmark for the Program on both of these primary summative measures has been set at 75%. This figure represents the average percentage of students successfully meeting both of the measures for a behavior. This benchmark has been determined by all faculty.
Table 1 – Plan for Assessment of Individual Behaviors with Competencies
Achievement Measure of Field Experience –Endpoint (AMFE-E)
The AMFE-E is the first of the Social Work Program’s two primary, direct, summative measures of Explicit Curriculum. This tool measures student performance on each of the forty-one behaviors that operationalize the ten core competencies. Students take the pre-formatted tool and personalize it to their field agency and by developing at least two Learning Activities related to each behavior. These are written by the student in the form of SMART (Specific, Measureable, Attainable, Realistic, Timely) objectives. This process helps the student integrate and link knowledge, values, and skills necessary for attainment of behaviors and ultimately core competencies.
The first administration of the Achievement Measure of Field Experience happens at the Midpoint (December) of the field experience. This involves reviewing the student’s progress on their Learning Activities, as well as matching the descriptive standards on the measurement tool to the student’s performance during the field experience. The Endpoint (April) administration of this tool is completed by the Field Instructor, providing student, and Avila faculty with valuable feedback from a rater outside of the classroom, and working in a setting critical to student’s ultimate success. Aggregate data from the Endpoint collection using this tool informs student attainment of the Program’s core competencies.
As a formative measure, the AMFE – M results assist the student and the Field Instructor in noting the student’s own perceptions of their strengths and needs, how those compare to the Field Instructor’s ratings, and offer data to guide the necessary planning around development or modification of Learning Activities for the next semester in field. Students use the tool for self-evaluation at Midpoint and can thus compare and contrast their perception of their demonstrated achievement of knowledge, values, and skills with the evaluation of their Field Instructor.
The Field Instructor rater uses a 0 - 4 point scale to rate levels of attainment of the professional behaviors related to nine competencies. The behaviors associated with each specific competency are aggregated and averaged to produce the final rating for the Program’s level of attainment in student achievement. The Benchmark designated is 3.0. This was attained and exceeded in all Competencies this year. Faculty will utilize this data in continued development of courses.
Achievement Measure of Capstone (ACMR)
The Achievement Measure of Capstone (AMC) serves as the Program’s second primary measure. This measure utilizes a fixed measurement scale with corresponding levels of achievement and incorporates a standard description of exemplars for each level of attainment related to behaviors as follows:
0 = Unacceptable
1 = Insufficient
2 = Emerging
3 = Capable (Program goal)
4 = Strong
Utilizing this scale as the guiding framework, the AMC is comprised of a set of rubrics that, taken in their entirety, measure all core competencies and behaviors. As every part of the Capstone Project measures a different subset of behaviors, each part of the Capstone Project has its own rubric. This equates to five rubrics total for the following parts of the assignments: Prospectus; Part 1: Problem Identification & Data Collection; Part 2: Micro/Macro Intervention; Part 3: Evaluation; and Part 4: Discussion. As an example, Part 1: Problem Identification and Data Collection, integrates corresponding behaviors, and each behavior is evaluated utilizing the 0-4 scale and the standard description of exemplars outlined above. An external evaluator, who is a practicing social worker with a Master’s degree in Social Work, assesses student achievement on the Poster and Presentation at the Colloquium event, based on a rubric utilizing the 0-4 scale.
Individual Student Ratings
During an academic year, each student is rated individually on their attainment of each Behavior, using each of the two measures.
Aggregate of Student Ratings = Behavior Attainment by two Measures
Next, for each of the two measures, for each Behavior, all individual student ratings are averaged. This aggregation results in two rates (%) of student attainment for each distinct Behavior.
Aggregate of two measure’s ratings = Program’s Attainment of Behaviors
In order to obtain the Program’s final percentage of student attainment of a distinct Behavior, the aggregate scores calculated for each of the two measures, are then averaged for each Behavior. This computation provides the Program’s percentage of student attainment for each Behavior, as indicated through the implementation of a multimodal assessment.
Aggregate of Behavior Ratings = Program’s Attainment of Core Competencies
Finally, in order to calculate the Program’s student attainment rate (%) for each Core Competency, the Program’s student attainment rate for each Behavior is used. All Behaviors related to a distinct Core Competency provide a student attainment rate that is averaged with that of all of the Behaviors clustered under that Competency. This calculation produces a final student attainment percentage for that Competency. This process is repeated for each Competency.
Benchmark for the Outcomes
BSW Program faculty have agreed to set the Benchmark level at a 3 out of 4 or 75% or above achievement rate for all tools that measure the Explicit Curriculum and the Implicit Curriculum. Part of the consideration for these high levels of attainment is that students are required to have a 2.5 GPA and a minimum of “C” in all Social Work courses in the Avila Social Work Program in order to be retained in the Program. However, the faculty are not satisfied with graduating a lot of “C+” level Social Workers. Faculty generally agree that Program expectations are at the “B” to “A” level for the majority of our students. Faculty want students to advance to higher-level cognitive skills related to their Social Work values, knowledge, and skills. Faculty expect students to move beyond knowledge, and comprehension, to demonstrate analysis, synthesis, and evaluation at an appropriate undergraduate level. Avila has established a regional reputation that their Social Work Program graduates demonstrate a high level of performance in the requisite competencies for the field of Social Work. The established Benchmarks help students and faculty continue to strive to excel. In the spirit of respecting all individuals, Social Work faculty recognize that for some students a “C” may be their best level of achievement, and we embrace that awareness, while also encouraging that student to attain their highest potential possible during their time in the Social Work Program.
Data Analysis and Continuous Improvement as a Result of Assessment
Program attainment of the ≥ 75% benchmark on all core competencies and almost all behaviors demonstrates that overall, the Avila Social Work Program is achieving its goals and mission at a high level. All students demonstrated capability or higher on their Capstone Projects. As a result of the assessment process, the following changes will occur:
- The Sheafor & Horseji text book used in SW 498 will also be used in SW 497 to assist students in identifying intervention strategies as they identify a problem, goals, and conduct their literature reviews in SW 397.
- The instructors from SW 397 and SW 498 will look at how to ease the transition from the Research Proposal to the Capstone Project Prospectus.
- SW 462 family assignment will serve to model application of evidence-based models as intervention strategies, as well as population and social problem research as rehearsal for what will occur in SW 498 in the context of the Capstone Project.
Recent Results of Assessment Activities
As the chart below indicates, based on evaluation of last year's assessment activities, Social Work graduates at Avila had attained all of the Core Competencies at the Benchmark rate by the time they completed their Social Work courses.
Publication of Assessment Results -
This Assessment Summary Report to all program constituencies: students, faculty, administration, Social Work alumni, Program advisory board, and others in the Kansas City regional community is updated annually on the Program web page.
Avila BSW Core Competency Achievement
Core Competency Achievement