Prevention Workforce Evaluation

Integrated Primary Prevention Activity Tracker Draft 5-9-23

Integrated Primary Prevention Activity Tracker (I-PAcT)

Contents

Installation name and Service branch 1

Introductory prompt 1

Preventionist time 2

Activity description 2

Domains and Risk and Protective Factors 3

Participation 4

Implementation status 5

Ecological levels 5

Planning 5

Needs assessment 6

Adaptations 6

Dosage 6

Evaluation 7

Sustainability 7

Respondents: One response per IPPW team.

Frequency: Every six months

Integrated Primary Prevention Activity Tracker items

Notes on purpose and source if applicable

Metric

Installation name and Service branch

[installation information tracked via survey link identifiers]

N/A

Introductory prompt

[First screen]

OMB CONTROL NUMBER:  0704-0644

OMB EXPIRATION DATE:

AGENCY DISCLOSURE NOTICE

The public reporting burden for this collection of information, 0704-0644, is estimated to average 60 minutes per response, including the time for reviewing instructions and completing and reviewing the collection of information. Send comments regarding the burden estimate or burden reduction suggestions to the Department of Defense, Washington Headquarters Services, at whs.mc-alex.esd.mbx.dd-dod-information-collections@mail.mil. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to any penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number.

Please read:

The following questions ask about integrated primary prevention activities conducted in the last six months. Please include all prevention activities that were implemented in the last six months, even if they were discontinued during that time period.

A prevention activity is a single activity or closely related series of activities whose purpose is to stop harmful behaviors before they occur. They may be policies, programs, or practices. The organization’s comprehensive integrated prevention plan should include the prevention activities included in this tracker, but activities should not be omitted from this tracker if they are not included in the prevention plan.

Note: Efforts that target reporting, treatment, or follow-up after a harmful behavior has occurred are not considered ‘primary’ prevention activities and should not be reported in this tracker. The only exception is suicide postvention activities, which aim to reduce risk to and prevent harmful outcomes for those impacted by suicide loss.

N/A

Preventionist time

1. For each of the Integrated Primary Prevention Workforce personnel at your installation that worked on planning, implementing, or evaluating prevention activities in the last six months, please provide the following information:

Please include information for individuals even if they are no longer at your location if they worked on prevention activities at your installation in the last six months.

Include all time spent planning a prevention program, advising leadership on a prevention topic or attending professional development on a prevention topic as percent effort for the relevant prevention domain (e.g., child abuse). If time spent is unrelated to a prevention domain, count this as % effort under “all other NON-prevention work”.

• Preventionist Title

• GS Grade

• GS Step (1-10)

• % effort spent on each prevention domain:

  • Child abuse

  • Domestic abuse

  • Retaliation

  • Sexual assault

  • Sexual harassment

  • Other harassment (e.g., hazing, bullying, race/ethnicity harassment)

  • Suicide (e.g. ideation, attempts, and deaths)

  • All other NON-prevention work

[completed in table format, % effort sums to 100%)

Will be used for Task 8, Cost-Benefit Analysis

Domains are from list provided by Andra Tharp on 1/5/23

4.1.1; 4.1.2; 2.2.4

Activity description

2. Please enter the name and a brief description of each activity you/your prevention team has planned, implemented, and/or evaluated in the last six months. Include any activities that were discontinued in the last six months.

	Activity name	1-2 sentence description	Activity type. [Link to activity type descriptions]	Is this activity evidence-based?
Activity A	[Open text]	[Open text]	[drop down list]	[drop down Yes and No – if yes, free text to provide a link to a study where it was shown to be effective]
Activity B
…

Activity Type Descriptions

[This list to be provided as link and provided ahead of time for completing Activity Type question above.]

Skill development	Interactive skill-building workshop or session that primarily involves active participation from all attendees, including activities such as practicing skills, role-playing, teamwork, and/or group discussions. May be one-time or multi-sessions. For training sessions that are not primarily interactive or focused on skill building and primarily involve one-way transmission of information from speaker to audience, such as lecture-based presentations or standard trainings, recorded video content, and Commander’s Call talking points, please select “Other” for Activity Type and describe. A multi-session program would be considered one activity.
Social/emotional care and support	Programs or activities that provide social and emotional support to Service members and their families (e.g., social connectedness interventions)
Media campaign	Coordinated distribution and reinforcement of messages and materials related to a prevention topic. Often intended to share information or change opinions, attitudes, or norms about a prevention-related topic, but may have other goals as well. Media campaigns typically include coordinated efforts to test messaging, target materials to specific audiences, and strategically disseminate materials where they will be most impactful. Materials may include posters, social media posts, radio ads, billboards, newsletters, flyers, text messages. For single activities that are not part of a coordinated media campaign (such as prevention-themed walks, fun runs, pancake breakfasts, and Teal Ribbon Weak activities), please select “Other” for Activity Type and describe. Each campaign is considered a separate prevention activity
Policy	Efforts to change or consistently enforce existing military policy, or create new policy to prevent harmful behavior before it occurs. Each policy is considered a separate prevention activity
Environmental strategy to establish protective environments and healthy climates	Efforts to prevent harmful behaviors by altering physical environments or social climates Examples include increasing leadership supervision for high-risk on-base locations and across digital communications; modifying environments to reduce access to lethal means; and efforts to alter social norms or command climates that are not captured by other activity types. Each strategy is considered a separate prevention activity.
Community collaboration activities	Efforts to work with the local community to change conditions to make harmful behavior less likely to occur. Activities to influence, support, or reinforce community partner practices This may include impacting local policy, participating in community prevention activities or events, or partnering with community organizations to improve care coordination. Each coordinated collaborative effort is considered a separate prevention activity.
Other, please specify*	Some other activity not described above.

*Activities labeled as Other will be asked Question 3, Domains and Risk and Protective Factors, but will skip all other questions

Component 3. Adhered to best practices

Component 5. Consist of multiple activity types

Used to assess Quality Implementation (type/quality of prevention activity)

2.3.1; 2.3.2; 2.5.1

Domains and Risk and Protective Factors

3. Which of the following does this activity address? Please select all that apply.

Prevention domains

1. Child abuse

2. Domestic abuse

3. Retaliation

4. Sexual assault

5. Sexual harassment

6. Other harassment (e.g., hazing, bullying, race/ethnicity harassment)

7. Suicide (e.g. ideation, attempts, and deaths)

Comprehensiveness Component 2. Targeted the most commonly encountered forms of interpersonal and self-directed harm at their installation. This assessment will compare the stated goals of their prevention activity from the prevention activity tracker to the needs indicated in the installation’s DEOCS data.

Comprehensiveness Component 3. Targeted multiple harmful behaviors

Comprehensiveness Component 4. Targeted multiple risk and protective factors

Domains are combined from DODI 6400 and SPARX – see crosswalk

Used to assess comprehensive approach (cross-cutting domains)

2.3.1; 2.3.2; 2.5.1

RISK/PROTECTIVE FACTORS RELATED TO (some could be risk or protective)

• Unit or workplace climate (e.g., fair, inclusive, hostile, respect, cohesive)

• Leadership (e.g., passive, toxic, supportive, transformational)

• Everyday stressors (e.g., parenting, chronic pain, legal problems, acculturation)/Coping skills

• Prior traumas (e.g., sexual, combat related)

• Social skills (e.g., problem solving skills, empathy, emotional regulation)

• Norms (e.g., level of acceptance of violence, hyper-masculinity, traditional gender role norms; social norms supportive or inhibitive of SV and male sexual entitlement)

• Use of alcohol or other substances (e.g., binge drinking)

• Mental health (e.g., history of depression)

• Hope/hopelessness or morale

• Relationships (e.g., peers, family, partner) problems (e.g., loss of relationships, isolation)/Social connectedness (e.g., feeling connected to school, community, and other social institutions; support from partners, friends, and family)

• Access to lethal means/Safe storage of lethal means

• Consistent access/lack access to high quality behavioral health and medical services

• Finances/employment

Risk and protective factors

From Andra Tharp 1/5/23:

DEOCS Risk and Protective Factors

CDC Technical Packages

• Suicide

• Sexual violence

• Intimate partner violence

• Child abuse

2.3.1; 2.3.2; 2.5.1

Participation

Not asked for media campaign, policy, or environmental strategy

Rank	How many individuals participated* in the prevention activity in the last six months? *Participated means completed at least 75% of the activity. Please be as precise as possible.	In the last six months, how many hours did the average participant spend participating in the prevention activity? Include all time spent attending an activity, completing remote activities, and participating in booster sessions.
E1-E4
E5-E6
E7+
O1-O3
O4-O6
O7+

Used for Task 8, Cost-Benefit Analysis

4.1.1; 4.1.2

Implementation status

6. Which of the following best describes the implementation status of this activity?

1. Initial exploration and adoption of activity

2. Planning for implementation (e.g. drafting workplan, hiring staff)

3. Initial implementation (e.g., pilot testing, initial roll-out with a limited scope)

4. Full implementation

5. Innovation and quality improvement following full implementation

6. Sustainability phase

7. Discontinued

Status

2.5.4

IF g. DISCONTINUED:

6. Please indicate the reason for discontinuing the prevention activity: [select one]

1. Activity was not data-informed, research-based, or evaluation results showed it was not achieving desired outcomes

2. Activity was scheduled to end (i.e., not a continuous activity)

3. Other: Please describe the reason for discontinuation:____________

Used to track if programs have been discontinued and qualitatively track reasons for discontinuation (no explicit “scoring”

???

Ecological levels

6. Which of the following ecological levels are the target(s) of this prevention activity?

1. Individual (i.e., personal factors that increase the likelihood of becoming a victim or perpetrator of violence. Some of these factors are age, education, income, substance use, or history of abuse. Prevention strategies at this level promote attitudes, beliefs, and behaviors that prevent violence. Source: CDC)

2. Interpersonal/Relationship (i.e. close relationships that may increase the risk of experiencing violence as a victim or perpetrator. A person’s closest social circle-peers, partners and family members-influences their behavior and contribute to their experience. Prevention strategies at this level may include family-focused prevention programs and mentoring and peer programs designed to strengthen communication, promote positive peer norms, problem-solving skills and promote healthy relationships. Source: CDC)

3. Community/Organizational (i.e., intervening on characteristics of different settings, such as workplaces or neighborhoods, that are associated with becoming victims or perpetrators of violence. Prevention strategies at this level focus on improving the physical and social environment in these setting. Source: CDC).

Comprehensiveness Component 1. Were implemented across ecological levels (individual, interpersonal, and community/ organizational).

Used to assess Comprehensive approach (PPOA 2.0) – multiple ecological levels across activities

2.4.1; 2.5.1

Planning

6. In the last six months, which of the following tasks, if any, have you/your team engaged in to plan this prevention activity? Select all that apply.

1. Held a meeting to coordinate implementation tasks

2. Actively involved stakeholders (i.e., individuals outside of the prevention team) in planning activities (for example, by inclusion in working groups)

3. Collaborated with another office or department to plan or implement this activity

Quality Component 2. Systematically planned

Used to assess Quality implementation

- Teaming (CFIR 2.0)

2.3.1; 2.3.2; 2.5.1

4. Searched for research evidence on this activity

5. Reviewed research evidence on this activity

6. Developed and/or revised a systematic workplan detailing implementation tasks and timeline

7. Created a budget for the activity

8. Secured funding for the activity

9. Identified (hired, trained, or assigned) staff responsible for implementing the prevention activity

10. Identified (hired, trained, or assigned) staff responsible for evaluating the prevention activity

11. Worked on a plan for monitoring the implementation process (i.e., plans to evaluate inputs, activities, and outputs; also called a process evaluation). This may include information about the dosage, reach, participant and staff perceptions, and quality (or fidelity) of implementation.

12. Worked on a plan to evaluate outcomes of the prevention activity (i.e., plans to evaluate the short, intermediate or long-term changes in participant knowledge, attitudes, skills, behavioral intentions, and/or behaviors).

13. Worked on a plan to sustain the prevention activity

Used to assess Quality implementation

• Planning (CFIR 2.0)

• Comprehensive approach (PPOA 2.0)

• Quality implementation (PPOA 2.0)

• Continuous evaluation (PPOA 2.0)

2.3.1; 2.3.2; 2.5.1

14. Pilot-tested the activity, implemented it in small steps, or conducted trials to test aspects of the activity

• Doing (CFIR 2.0)

2.3.1; 2.3.2; 2.5.1

Needs assessment

6. In the last six months, which of the following types of information, if any, did you/your team collect and use to inform planning or implementation of this prevention activity? Select all that apply.

1. Priorities, preferences, and needs of staff/individuals implementing the activity

2. Priorities, preferences, and needs of leadership

3. Formal data on the needs of the target population

4. Informal information on the needs of the target population

5. Needs specific to the target population(s) with regards to diversity

6. Needs specific to individuals in the target population(s) with cross-cutting identities (e.g., racial minorities who are also sexual minorities)

Quality Component 1. Responsive to the needs of the installation

Used to assess Quality implementation

• Assessing needs (CFIR 2.0)

• Understanding the problem (PPOA 2.0)

2.3.1; 2.3.2; 2.5.1

7. Collected information about barriers and/or facilitators to implementing the activity

• Assessing context (CFIR 2.0)

2.3.1; 2.3.2; 2.5.1

8. Reviewed information from the Command Climate Assessment

2.3.1; 2.3.2; 2.5.1

Adaptations

6. Have you/your team EVER adapted or made modifications to the activity to alter it from its original design? [select one]

1. Yes, we have made changes to the activity

2. No, we are implementing an existing pre-packaged program exactly as-is

3. This activity was developed specifically for our use and not modified from an existing prevention program

If YES, have modified

12. Have you/your team EVER made any of the following changes to the prevention activity (select all that apply):

1. Adapted content to address a different prevention focus without consulting the developers of the program (for example, modifying a sexual harassment bystander intervention to address suicide prevention instead)

2. Removed substantive content, such as training modules on a prevention topic

3. Abbreviated the length of the activity

4. Removed interactive components

5. Removed opportunities to practice skills

6. Other, please describe________

Used to assess Quality implementation

• Tailoring Strategies/ Adapting (CFIR 2.0)

Dosage

ONLY ASKED FOR INTERACTIVE WORKSHOP/SKILL BUILDING SESSION

6. How many sessions or events does this activity consist of? Indicate the total number of planned sessions, including any booster sessions, that all or most participants are expected to attend, even if not all of the sessions were conducted in the last six months. [numeric value]

Quality Component 5. Of sufficient dose

Used to assess Quality implementation

2.3.1; 2.3.2; 2.5.1

6. Does the prevention activity include periodic reminder or booster sessions to reinforce core messages?

• Yes

• No

Quality Component 5. Of sufficient dose

Used to assess Quality implementation

2.3.1; 2.3.2; 2.5.1

Evaluation

6. In the last six months, which of the following tasks, if any, have you/your team engaged in as part of efforts to evaluate this prevention activity? Select all that apply.

1. Collected quantitative and qualitive information that indicates the quality of implementation (i.e., evaluated inputs, activities, and outputs; also called a process evaluation). This may include information about the dosage, reach, participant and staff perceptions, and quality (or fidelity) of implementation.

2. If yes to “a”: Discussed quantitative and qualitive information that indicates the quality of implementation (i.e., evaluated inputs, activities, and outputs; also called a process evaluation). This may include information about the dosage, reach, participant and staff perceptions, and quality (or fidelity) of implementation.

3. Collected quantitative and/or qualitative information to evaluate the outcomes of the prevention activity (i.e. the short, intermediate or long-term changes in participant knowledge, attitudes, skills, behavioral intentions, and/or behaviors).

4. If yes to “c”: Discussed quantitative and/or qualitative information to evaluate the outcomes of the prevention activity (i.e. the short, intermediate or long-term changes in participant knowledge, attitudes, skills, behavioral intentions, and/or behaviors).

Quality Component 4. Evaluated, and used results to inform activities

Used to assess Quality implementation

• Evaluation and reflection (CFIR 2.0)

• Continuous evaluation (PPOA 2.0)

2.3.1; 2.3.2; 2.5.1

Sustainability

6. Regarding sustainability of the activity, which, if any, of the following apply to this activity?

• Permanent staff have been assigned to implement this activity

• This activity has a champion within leadership

• This activity has transitioned from temporary or pilot status to permanent status

• This activity has a stable source of funding

• This activity has been assigned permanent physical space

• We expect to continue implementing this activity for some time

Used to assess Quality implementation

• Level of Institutionalization Measure (LoIN)

2.5.5

SCORING

Key components of quality implementation

• Component 1. Responsive to the needs of the installation

• Component 2. Systematically planned

• Component 3. Adhered to best practices (including not making adaptations known to decrease effectiveness; and of sufficient dose, as applicable)

• Component 4. Evaluated, and used results to inform activities

Key components of comprehensiveness

• Component 1. Were implemented across ecological levels (individual, interpersonal, and community/organizational).

• Component 2. Targeted the most commonly encountered forms of interpersonal and self-directed harm at their installation. This assessment will compare the stated goals of their prevention activity from the prevention activity tracker to the needs indicated in the installation’s DEOCs data and command climate assessment

• Component 3. Targeted multiple harmful behaviors

• Component 4. Targeted multiple risk and protective factors

• Component 5. Consist of multiple activity types