10: Case Tracking and Program Evaluation

Case Tracking

Case tracking involves collecting data about a case. Data can be collected from the various MDT members and/or from victims and offenders. The purpose of case tracking might be:

• To enable the MDT to analyze their caseload

• To measure the success of specific cases and the over-all effectiveness of the team

• To ensure that cases are being monitored

• For tracking the MDT case review meeting information

• To improve program performance (see Chapter 10 for a primer)

• For some forms of program evaluation (see Chapter 10 for a primer)

• To educate the public

• To create targeted outreach campaigns

• To identify patterns of behavior that might eventually be used to predict risk

• To provide evidence of effectiveness in grant applications to funding agencies
   

The length of case tracking may vary considerably from initial intake to some period of time after the close of the case. Often, data are retained in different departments and agencies and must be extracted in some fashion. Strategies for obtaining case tracking data include:

• Collecting information at case review

• Agency-completed forms that are returned to the MDT Coordinator

• Telephoning the agency directly for information

• Appointing a staff member (e.g., the victim advocate) to collect the information

• Some combination of these methods
   

Consider developing a form that captures this case-level data, sometimes referred to as a case tracking form. Discuss which data elements are important for the MDT to collect by identifying the purpose of the data.

Data tracking systems for storing such information might be as simple as Microsoft Excel or Access. Data points to consider collecting include:
 

Data Management Plan
 

If a case-tracking plan is adopted, the MDT will need to develop a data management plan. The plan not only increases accountability of the data, but also reduces the number of people who handle the data. Consider the following:

• Save all information on password protected computers.

• Use identification numbers for each person entering data.

• Incorporate periodic review of data forms for completeness and accuracy. Data checking can be accomplished by randomly selecting, for example, 10% of the cases. Compare the printouts of data entered with the original forms. Be sure to report the time and date of reviews and any conclusions. If problems are identified, bring these problems to the MDT to identify solutions.
   

Depending on the type of data being collected and the representative’s agency, some MDTs will need to obtain approval for data collection from their institutional review board (IRB).

Primer on Research and Evaluation

Evaluation
 

Since the 1970s, teams have become a popular mechanism for addressing a range of issues. However, sometimes teams are promoted due to their psychological value rather than their empirically validated benefit. The goal of developing a team is to support and respond to victims of elder abuse (a performance outcome) rather than the goal of developing a team. Therefore, the MDT will want to assess their ability to achieve this goal.

Evaluation of MDTs may occur at the level of:
 

Structure

Process

Outcomes

• Structure
  Who participates on the MDTs; what is the organizational affiliation of the MDT

• Process
  How is case review conducted; how are cases referred to the MDT

• Outcomes
  As a result of the MDT, are clients assigned guardianship; has the abuse stopped

Evaluation of the MDT and Clients
 

Strong evaluation helps protect the integrity of the program and can be a powerful tool for program sustainability. Evaluations can be used to:

• Promote the model of service delivery to funders and other stakeholders

• Serve as the basis for making changes in the program design

• Identify areas for professional development

• Determine new partners needed to strengthen the MDT

• Leverage results to obtain, retain, or expand funding

• Estimate the cost of the program for a cost-benefit analysis
   

The first step in an evaluation is developing a logic model. A logic model is simply a visual representation that describes how a program or intervention will work. It links the program’s activities to the planners’ goals and objectives, and it identifies anticipated short- and longer-term outcomes. As such, the logic model is the foundation for program evaluation, which in turn is essential for learning “what works” in victim assistance and compensation. Based on the logic model, a program evaluation plan can be crafted.

A well-crafted program evaluation is essential in determining whether the program is meeting its goals and what is producing the desired outcomes. For example, just prior to the Los Angeles Elder Justice Forensic Center being created, they had concluded that the MDTs that had developed in response to elder abuse cases were extremely large and it was difficult to actively work through a case, necessitating change.

Open Full Screen
 

Sample Generic Logic Model

Open Full Screen
 

Program Evaluation
 

There are different kinds of evaluation and different aspects of your program you may want to evaluate. For example:

• The functioning of the MDT

• Team performance

• MDT satisfaction

• Client satisfaction

• Board survey (if applicable)
   

Evaluation targeted at different points in the process will capture different experiences. For example, a client satisfaction evaluation immediately after the case has closed will capture different information than a client satisfaction survey administered six months after the case has closed.

Regardless of the type of program evaluation, the best evaluations engage program staff, volunteers, clients, and other major stakeholders in the design and implementation of the evaluation.

MDT Functioning and Satisfaction

At some point after the MDT has been established, you will want to evaluate the functioning and effectiveness of your MDT. You may want to focus your evaluation on the elements of a successful MDT. These characteristics will need to be quantified for evaluation purposes. For example:

Team Trust and Cohesion

• The team has a shared interdisciplinary team philosophy

• The team has honest and continuous communication

• The team readily shares knowledge (as opposed to information), for example, through informal and formal cross training

• MDT members are comfortable exchanging information

• There is a sense of collegiality among MDT members

• MDT members share ideas and experiences, discuss cases, and engage in a critical analysis of cases

• There is a shared belief that working as a team leads to better outcomes

• Mutual support is provided by MDT members

• MDT members feel mutual trust and respect

• MDT members are able to develop trust with victims

• MDT members complement each other’s functions

• MDT members share resources

• MDT members enhance each other’s capacity to address a crucial victim need

• The agencies represented on the MDT have changed the way they operate as a result of participation on the MDT
   

Administrative Functions of the MDT

• Roles and responsibilities among partner agencies and individuals are clearly defined, typically in writing through MOUs or IAAs

• The MDT has adequate financial support

• The MDT has a written financial plan and a clear strategy for obtaining financial resources with identified responsibilities for implementing it

• The MDT has strong leadership including high-level, visible leaders

• Protocols have been adopted by the MDT

• There is adequate space and support dedicated to the MDT

• There is protected time for MDT members (e.g., regularly scheduled case review meetings)

• Professionals represented on the MDT are from a diverse range of disciplines that reflect the needs of the community

• The MDT Coordinator provides strong leadership

• The MDT Coordinator is accountable to the MDT

• There is opportunity for ongoing education and training for MDT members

• MDT members attend meetings regularly

• The MDT members review and evaluate their program regularly

• The MDT has clearly articulated goals, strategies, and indicators of progress that provide a sense of direction

• Evaluation results are used to enhance future efforts

• The MDT has established evidence of progress in affecting desired outcomes

• The MDT is exposed to some media coverage

• Joint documentation is utilized, for example, all members sign reports
   

Victim Satisfaction
 

The challenge associated with victim satisfaction surveys is that victims do not always perceive the “process” the way the systems perceive the process. While the MDT may perceive the investigation, services, and case review as a seamless process, victims may want to rate those activities individually. Victim satisfaction surveys may want to address whether:

• Victims receive follow-up medical care where necessary

• Victim are able to access Victims of Crime Act financial support

• Victims are able to get their medical bills paid

• Victims are able to secure safe housing

• Victims are able to build a sustainable support network

• Victims are satisfied with their interactions with various team members

• Victims are satisfied with their- intervention
   

Lessons Learned
 

Consider keeping a Lessons Learned log that you can share with the MDT at annual or semi-annual review meetings. The MDT Coordinator can be the keeper of the log, but have MDT members provide suggestions for the log.

Evaluation Logistics

Frequency of Administration of Surveys
 

The MDT will need to decide how frequently to administer various surveys: After every meeting, every six months, annually.

Instruments
 

There are a number of surveys that might be adapted for the purposes described above. However, there are no empirically validated measures of client satisfaction in the context of elder abuse MDTs. Where feasible, consider partnering with a university faculty member or graduate student.

Data Collection Plan
 

As part of your evaluation plan, a plan for collecting and storing data will need to be developed to ensure information is being captures that allows the evaluation questions to be answered. Several data collection plans exist.

Seek Out Consultations
 

Consult with individuals who have considerable experience with MDTs, either via websites or through literature searches.[

Utilize Evaluation Results
 

Utilize the information obtained from these evaluation efforts to improve your program. It may be hard to hear that all of your efforts fail to result in perfect outcomes, but keep in mind that improvement is always possible and is definitely desirable.

Research: MDTs Make a Difference

There is very little research available on elder abuse collaborations such as MDTs. What is available is reviewed below.

• Empirical evidence suggests that a social worker-lawyer collaboration is more effective at reducing risk of elder abuse compared to a social worker alone.

• In an evaluation of the Los Angeles County Elder Justice Forensic Center (EJFC), the use of a MDT funneled more cases to the District Attorney, therefore there were more convictions in absolute numbers, but not a statistical improvement compared to cases managed solely by APS. That is, the EJFC had significantly more financial exploitation cases referred to the DA compared to APS cases (22% vs. 3% respectively), although the number of cases with charges filed (73% vs. 86% respectively) and the number of convictions (92% vs. 100%) were not significantly different.

• One study found that social workers made therapeutic referrals most frequently, although legal interventions showed the greatest improvement in terms of stopping the abuse. This suggests the need for multiple systems to work collaboratively. However, the study found that the more interventions implemented, the lower the rate of improvement, possibly because cases requiring more interventions are more complex and intractable.

• There are potentially multiple explanations for a condition indicating elder abuse, requiring greater interactive information gathering and fact checking in these cases. Using adult protective services data from two counties in Maryland, it was learned that social workers were more likely to substantiate physical abuse, neglect by others, and financial exploitation (but not self-neglect) compared to when a social worker and nurse worked a case collaboratively, suggesting a nurse persuaded the social worker the injury was a result of an accident rather than abuse. However, the collaborative approach did result in reductions of risk for physical abuse, neglect by others, and self-neglect (although not financial exploitation). However, recidivism rates did not differ. Nurses tended to focus more on health needs and functional abilities, while social workers asked more extensive questions about social needs, including relationships with family and support services. The authors concluded, however, that the marginal gains did not justify the costs associated with collaborative responses.

There is more research on the process of MDTs than outcomes. While we believe the benefits of collaboration carry over to victims, there is little empirical evidence one way or the other.