Prevention Workforce Evaluation

Summary of Leader and Supervisor Interview Protocols

Notes:

• Protocols are designed to be used at both baseline and follow-up interviews, and are designed flexibly to be adapted based on how components structure the prevention workforce.

• Protocols were designed to align with the Task 4 cohort study and provide additional insight into how both the prevention workforce leadership chain and other senior leaders view their role.

• Components of these protocols were designed to be complementary to other data collection efforts.

Interview Topic	Protocol #1: Operational Level Prevention Personnel	Protocol #2: Strategic Level Prevention Staff	Protocol #3: Prevention Collaboration Forum and Other Leaders
Background	X	X	X
Role in Prevention (tailored questions by level/role)	X	X	X
Understanding of prevention	X	X
Interaction with prevention workforce	X	X
Interaction with other stakeholders		X	X
Budgeting / planning process	X	X
Evaluation of prevention personnel efforts	X	X	X
Assessment of Impact	X	X	X
Overarching barriers and facilitators	X	X	X
Perceived differences in perspectives	X	X

INTRODUCTION/CONSENT TO BE READ BEFORE ALL PROTOCOLS:

Hello, my name is ___________ and I am a researcher at RAND Corporation, which is a non-profit, policy research organization. We are working with Office of the Secretary of Defense for Personnel and Readiness in collaboration with the Office of Force Resiliency’s Violence Prevention Cell to evaluate the Department of Defense’s integrated primary prevention workforce initiative. Thank you for taking the time to talk today. Our questions today are going to focus on your role supporting integrated primary prevention and strengths and challenges you have encountered in that role.

Before we begin, I want to assure you that your answers will be held in confidence by RAND to the extent allowed by law and DoD policy. [If more than one person in the virtual ‘room’: However, there are others on the line so please do not say anything you do not want others to know.] We are speaking with a variety of leaders and supervisors for the integrated primary prevention workforce. Findings from across these sources will be reported in briefings and reports to DoD, but we would not be naming you specifically in any of those documents. We will be taking notes during this discussion, but they will not have your name on them. There should be no discussion of open cases or investigations so as to respect the privacy of individuals who may participate in integrated primary prevention activities at this site.

Your participation is voluntary. Refusal to participate will involve no penalty or loss of benefits to which you are otherwise entitled. You may discontinue participation at any time without penalty or loss of benefits, to which you are otherwise entitled. If you have questions about your rights as a participant or need to report a participation-related injury or concern, you can contact RAND's Human Subjects Protection Committee using the contact information we emailed you about this study.

Any questions before we begin?

Task 5 Protocol #1: Operational Level Prevention Personnel

Audience: Leaders at the Operational Level. For example, prevention program managers who develop and oversee prevention program activities and budgets, determine and provide resources to meet identified need areas, and provide tools and technical assistance to prevention professionals at the tactical level. (NOTE: May run these as focus groups.)

1. Can you start by telling me a bit about yourself – what is your specific title, and how long have you been in the role? 

2. How is the prevention workforce in your organization organized, and how are you applying the prevention approach? (Note: This could be either all IPP Personnel or the IPPW specifically, as the respondent prefers.)

1. How are IPP teams composed here at the operational level?

2. How do you see prevention personnel fitting into the existing system / process?

[Note: Consider having visual of process/system from PPOA 2.0 available during discussion.]

3. What do you see as the most significant risk factors and harmful behaviors that prevention personnel should be addressing in your organization?

1.9.2

3. What types of tools and technical assistance do you provide to prevention personnel at the tactical level?

1. Do you do this proactively? Or do prevention personnel request support only when needed?

1.9.1

4. How have you helped to translate or apply the strategic guidance from DoD and [your DoD component] to your work or to the work of the tactical prevention personnel?

1. Examples could include: provided brief summaries of the integrated prevention policy or PPoA 2.0, provided easy to understand guidance on how to complete various steps in the prevention process (design, implement, evaluate prevention); summarized best practice approaches that sites could use.

1.1.1

1.1.2

5. Can you describe what data sources you use to prioritize prevention efforts and gauge resource needs?

1. What are the prevention needs of service members (in your component / at the tactical level / in your major command)?

6. How do you make budgeting or resource allocation decisions for prevention personnel or their activities (e.g., facilities, supplies, licenses, money) at the operational level?

1. Can you share the budget with us?

7. How do you make budgeting or resource allocation decisions for prevention personnel or their activities (e.g., facilities, supplies, licenses, money) at the tactical level?

1. Can you share the budget with us?

1.5.2

8. What role, if any, have you had in orienting or onboarding new prevention personnel?  

1. Are there any component-specific standard orientation/onboarding processes or materials for new prevention personnel? If so, what was the approach in designing them? Were you directly involved in creating the materials / in what way? Are there materials we may review?

9. 

   1.2.1

   1.2.2

   What challenges and successes have you faced with incorporating the new prevention staff at the operational level?

1. And at the tactical level?

1.1.1

1.1.2

10. Do you receive briefings, quarterly updates, or other types of information about the prevention staff or their activities?

1. If so, what do you receive and how is it provided? 

2. What do you do with that information?  

3. May we have any samples or blank templates as examples? 

1.1.1

1.1.22

11. How do you interact with the prevention personnel at the tactical level (e.g., periodic VTCs or webinars)?

1. Please describe these interactions and outcomes (e.g., goals, activity, feedback thus far, lessons learned).  

2. Do they ask you for help or resources?

3. Do you provide feedback on their plans, activities, or processes?

12. 

    1.3.1

    What impact of the new prevention personnel at the operational level do you anticipate (have you seen)? How do (will) they contribute to prevention across the enterprise?

1. Probe for types of impacts: frequency and quality of prevention efforts, reach, ability to integrate violence prevention efforts, improved data collection and monitoring.

1.3.1

13. What impact have you seen for service members thus far from the work of prevention personnel at the tactical level? How were you able to identify that impact? 

1. Probe for types of impacts: frequency and quality of prevention efforts, reach, ability to integrate violence prevention efforts, improved data collection and monitoring.

1.2.1

1.2.2

14. What barriers do you see or anticipate to incorporating the new prevention staff at the tactical level?

1. Probe for types of barriers (views about prevention activities, challenges adapting activities, funding, resources, collaboration, policies).

1.2.1

1.2.2

15. What facilitators do you see as potentially supporting this effort?

1. Probe for types of facilitators (leadership engagement, local champions, knowledge and beliefs about prevention, supportive policies).

1.1.1

16. We understand that you should have some functional oversight from the strategic level prevention personnel for your component – how and how often do you interact? How has that helped you to do your job better, if at all?

1.1.2

17. Have you encountered any differing perspectives between the installation and HQ level management and oversight on integrating the new prevention personnel and their activities?

1. If so, what were they? Were they complementary or at odds? How have those differences been addressed (if they have)? 

Task 5 Protocol #2: Strategic Level Prevention Personnel

Audience: Leaders at the Strategic Level. For example: Military Department Prevention Director / Program Head, Prevention Researcher / Prevention Policy Analyst.

1. Can you start by telling me a bit about yourself – what is your specific title, and how long have you been in the role? 

1. How do you interact with prevention personnel at the operational and/or tactical levels?

2. How is the rest of the IPP team at the strategic level composed?

1.9.1

2. How do you use prevention research to develop training materials?

3. 

   1.9.1

   Can you describe how you synthesize and translate and/or disseminate prevention policy and research to operational and tactical IPP personnel?

1. (Note: This could include best practice summaries, DoD or component-specific policy, and military-specific or broader research).

4. 

   1.1.1

   1.1.2

   How do you use data to prioritize prevention efforts and gauge resource needs?

1. How was your strategic plan informed by that data? What can you tell me about your strategic plan? Can you provide a copy to us?

2. What do you see as the most significant risk and protective factors for servicemembers? (Note: Respondents can answer for their military department/service or overall.) Why?

3. How do you assess the implementation and evaluation of prevention activities?

1. What unit of analysis do you use when evaluating outcomes (aggregated by unit or installation, component-wide, other, a combination of different views)?

5. How did/do you make budgeting or resource allocation decisions for prevention personnel or their activities (e.g., facilities, supplies, licenses, money)? 

1.1.1

1.1.2

6. What policies do you have or have you made specifically about prevention staff responsibilities, roles, policies, procedures, processes, etc.?

1. How you envision these will contribute to more integration across prevention efforts?   

7. 

   1.7.3

   Did you have/are you having input into creating prevention job positions? And making hiring decisions? If yes, what did you consider when creating job titles and descriptions and making hiring decisions? Did you need to make any adjustments to their original approach?  

1.5.2

8. What role, if any, have you or your HQ-level staff had a role in orienting or onboarding new prevention personnel?  

1. Did your team develop standard orientation/onboarding processes or materials for new prevention personnel? If so, what was the approach in designing them, and are there materials we may review?

1.2.1

1.2.2

9. What challenges and successes have you faced in fulfilling your role?

1.2.1

1.2.2

10. What challenges and successes have you faced with incorporating the new prevention staff?

1.1.1

1.1.2

11. Do you receive briefings, quarterly updates, or other types of information about the prevention staff or their activities?

1. If so, what do you receive and how is it provided? 

2. What do you do with that information?  

3. May we have any samples or blank templates as examples? 

1.1.1

1.1.22

12. Do you or your HQ-level staff interact with the prevention personnel (e.g., periodic VTCs or webinars)?

1. Do you have visibility on their activities?

2. Please describe these interactions and outcomes (e.g., goals, activity, feedback thus far, lessons learned).  

2.1.2

13. Who are the other strategic-level stakeholders / partners with whom you collaborate outside your organization? [Assess strength of relationship using OSIE Integration scale, for each named relationship]

2.1.2

14. Can you describe your relationship with strategic leadership outside the prevention workforce (e.g. military commanders, P&R leadership, M&RA leadership)? [Assess strength of relationship using OSIE Integration scale, for each named relationship]

1.3.1

15. What impact of the new / expanded prevention workforce do you anticipate? Why?

16. 

    1.3.1

    What impact have you seen thus far from the prevention staff’s work at the tactical level? How were you able to identify that impact? 

1. Probe for types of impacts: frequency and quality of prevention efforts, reach, ability to integrate violence prevention efforts, improved data collection and monitoring. 

17. 

    1.3.1

    What impact have you seen thus far from the prevention staff’s work at the operational level? How were you able to identify that impact? 

1. Probe for types of impacts: frequency and quality of prevention efforts, reach, ability to integrate violence prevention efforts, improved data collection and monitoring. 

1.2.1

1.2.2

18. What barriers to you see or anticipate to incorporating the new prevention personnel at the tactical level?

1. Probe for types of barriers (views about prevention activities, challenges adapting activities, funding, resources, collaboration, policies)

1.2.1

1.2.2

19. What facilitators do you see as potentially supporting this effort?

1. Probe for types of facilitators (leadership engagement, local champions, knowledge and beliefs about prevention, supportive policies)

1.1.2

20. Have you encountered any differing perspectives between the installation and HQ level management and oversight on integrating the new prevention workforce and their activities?

1. If so, what were they? Were they complementary or at odds? How have those differences been addressed (if they have)? 

Task 5 Protocol #3: OSD/Strategic Leaders (not Prevention Personnel)

Audience: Leaders at the Strategic Level who do not directly oversee prevention personnel or programming. For example: Members of the Prevention Collaborative Forum; leaders with a role in coordinating with prevention workforce in prevention activities.

Proposed OSD Level HQ Organizations

Personnel & Readiness

Force Resiliency

Defense Suicide Prevention Office (OFR/DHRA)

Sexual Assault Prevention and Response Office (OFR/DHRA)

Office of People Analytics (DHRA)

Office of Diversity Equity and Inclusion (OFR)

Office of Drug Demand Reduction (OFR)

Military Community and Family Programs (M&RA)

DoD Education Activity (M&RA)

Defense Human Resources Activity

Special Operations / Low-Intensity Conflict (SOLIC)

Joint Staff

1. Can you start by telling me a bit about yourself – what is your specific title, how long have you been in the role, and how long have you been participating in prevention activities (either as part of the Prevention Collaboration Forum or more broadly)? 

2. How would you describe the role and functioning of the Prevention Collaboration Forum and your participation on it? Probe on PCF activities related to:

1. Developing common definitions and collaborative strategies for prevention

2. Developing policies

3. Directing research

4. Aligning resources

5. Analyzing gaps

6. Synchronizing stakeholder activities

7. Conducting oversight on implementation of DoDI 6400.09 (Note: Update as needed)

3. How is your organization applying primary prevention principles?

2.1.2

4. At the operational and/or tactical level, what type of relationship is envisioned between members of the Integrated Primary Prevention Workforce and members of your organization?

1. (Ex: Informing, networking, coordinating, cooperating, collaborating; bring visual from SPARX training in case would be helpful)

1.3.1

5. What impact of the new / expanded prevention workforce, if any, do you anticipate on your staff / area of responsibility? 

1.3.1

6. What impact have you seen thus far? How were you able to identify that impact? 

1.2.1

1.2.2

7. What barriers do you think the prevention effort will face/is facing/has faced?

1.2.1

1.2.2

8. What facilitators do you see as potentially supporting this effort?

1.2.1

1.2.2

9. Do you have any suggestions for improving the rollout of the integrated primary prevention workforce?