Attachment A - State Official Interview Protocol

Attachment A. CCBHC Demo Eval State Official Interview Protocol_OMB.docx

Evaluation of the Certified Community Behavioral Health Clinic Demonstration in Accordance with the Bipartisan Safer Communities Act

Attachment A - State Official Interview Protocol

OMB:

Document [docx]
Download: docx | pdf

CCBHC Demonstration Evaluation State Official Interview Protocol

Unless otherwise noted, questions will be asked of all cohorts included in interviews in a given year.

Year 3 (Fall 2025 - CARES Act Cohort in DY5, BSCA Cohort 1 in DY2)

Obtain verbal consent to record

I. Introduction/warm-up

  1. We would like to learn about your current role(s)/position(s). What are your key responsibilities related to the CCBHC demonstration?

    1. For how long have you held this position?

  2. How has demonstration implementation been going for your state? What has been your greatest success and what has been your greatest challenge during the past year?

    1. [BSCA Cohort 1] What features of the CCBHC model have worked well so far? How have these affected the care provided and outcomes in your state?

  3. [CARES Act Cohort, BSCA Cohort 1] Has the state’s role in demonstration implementation evolved since we last spoke with you? Please describe if so.

  4. Does your state monitor CCBHCs’ adherence to the certification criteria in an ongoing way? How if so?

    1. Do clinics monitor adherence themselves?

    2. How does the state work with CCBHCs to ensure they continue to meet the criteria?

  5. Has the composition of the state-level team overseeing the CCBHC demonstration changed in the last year?

    1. Does the state currently have sufficient staff capacity to oversee the demonstration? Are there certain roles or aspects of demonstration oversight that could benefit from additional staff?

    2. Does the state engage consultants to support the demonstration? How have consultants complemented the state’s demonstration oversight team?

    3. Does the state plan to expand its demonstration team in the future?

  6. In general, have CCBHCs in your state been able to maintain adherence to the certification criteria since we spoke last year?

II. Access

A. Evaluation question: What successes and challenges did new states and CCBHCs encounter in improving access to care? How did they overcome challenges? [BSCA Cohort 1 ONLY]

  1. Could you please update us on state and CCBHC activities to increase access to care in the last year?

    1. What kinds of technical assistance or guidance have the state or other groups provided to support CCBHCs’ efforts to increase access (if any)?

    2. Has the state changed any requirements related to access since the demonstration began?

  2. Could you please share some of the key successes that demonstration CCBHCs have had in increasing access to care?

  3. What challenges have demonstration CCBHCs encountered in increasing access to care? [Potential probes]:

    1. Can you think of any statewide or regional challenges? What about challenges for specific clinics?

    2. Do you recall any activities that CCBHCs considered, but did not end up implementing due to challenges?

    3. Have CCBHCs discontinued any activities originally implemented to increase access? Why?

    4. Have there been any access challenges specific to certain populations? (For example, people with disabilities, experiencing homelessness, children/adolescents, Veterans,)

    5. How, if at all, is the state supporting the CCBHC(s) to address these issues?

    6. How are CCBHCs working to overcome these challenges? What have been some of the more successful strategies to date?

III. Quality and Scope of Services

A. Evaluation question: Could CCBHCs hire the staff they needed to implement the intended scope of services? What roles have been challenging to hire and retain?

  1. Has the state implemented any new requirements related to staffing in the past year?

    1. Why did the state make these changes?

  2. Have your state’s CCBHCs been successful in hiring the numbers and types of staff reflected in their staffing plans?

    1. Did CCBHCs’ community needs assessments identify any staffing needs?

    2. Have certain types of staff been more challenging to hire than others? Has this changed since the beginning of the demonstration?

    3. What factors have influenced CCBHCs’ ability to hire required staff?

    4. Has the state done anything to assist CCBHCs in hiring needed staff or addressing staff hiring challenges?

  3. [CARES Act cohort] Have your state’s CCBHCs been successful in retaining staff needed to implement the CCBHC model during the demonstration?

    1. [If yes]: What has helped clinics to retain staff?

    2. [If no]: What challenges have clinics faced in retaining the required staff? For example, have certain staff types been more difficult to retain than others?

      1. What caused these challenges?

      2. Has the state done anything to assist CCBHCs in addressing these challenges?

    3. Has clinics’ ability to retain staff differed at different points during the demonstration? What accounts for this variation if so?

    4. Have certain CCBHCs had more trouble retaining staff than others? What accounts for this difference?

B. Evaluation question: What was the quality of care provided to CCBHC clients? How did it compare to state benchmarks? Did quality of care change over time?

  1. [CARES Act Cohort] Could you reflect on your state’s experience collecting and reporting the CCBHC quality measures since we last spoke?

    1. What has hindered data collection and reporting?

    2. What has facilitated data collection and reporting?

    3. Have you calculated the measures following the specifications?

      1. If not, why?

  2. [CARES Act Cohort: Are the required quality measures / [BSCA Cohort 1: Do you expect the required quality measures will be] helpful for measuring and improving your state’s CCBHCs and the quality of the care that your state’s CCBHCs provide?

    1. Why or why not?

    2. Do your state’s CCBHCs collect or plan to collect and use the optional measures? What about additional measures not included as required or optional for CCBHCs? Why or why not?

  3. [CARES Act Cohort] Has the state seen marked improvement or worsening of any of the quality measures?

    1. Which measure(s)? [Interviewer note: have a list of quality measures available for reference]

    2. What do you think accounts for this change?

    3. Why do you think this measure/these measures improved relative to others? For example, has the state made specific investments that could have influenced this measure more than others?

  4. Reflecting on the CCBHC model overall, do you think it improves the quality of behavioral health care (compared to standard practice provided by other behavioral health providers in your state)?

    1. If yes: Which aspects of the model improve care quality the most? For example, have any specific evidence-based practices influenced the quality of care?

    2. [CARES Act Cohort] How has the quality of care provided by CCBHCs changed over time?

  5. Are there any CCBHC requirements that do not significantly improve the quality of care? Why? (For example, because they are too burdensome, not well-received, etc.)

  6. How, if at all, is the state supporting CCBHCs’ continuous quality improvement activities?

    1. How has the state used performance on quality measures to help clinics improve quality, if at all?

    2. Have the state or other groups offered any technical assistance related to quality?

  7. [CARES Act Cohort] Has the state used quality measure performance to change any policies? Please describe if so.

C. Evaluation question: How did states structure their quality bonus payment systems? Did state awards of quality bonus payments change over time?

  1. Have there been any changes to the way the state is awarding/plans to award quality bonus payments [CARES Act Cohort] in the last year/[BSCA 1 Cohort] since the state applied for the demonstration? For example, have there been any changes to measures or measure thresholds used or the extent reporting or improving on a measure factors into your approach?

  2. [CARES Act Cohort state(s) with QBP programs] Has your state awarded any quality bonus payments since the beginning of the demonstration?

    1. [If yes] Did you award them to all clinics or only some? What explains this?

    2. [If yes] Did you award them to the same clinics or different clinics? Is there consistency in which clinics receive them annually?

    3. [If yes] Do you think award of QBP reflects an improvement in quality or is it reflective of something else (e.g., the thresholds set for the measures)?

      1. [If reflective of quality] What specific mechanisms of the quality bonus payment systems changed provider behaviors? What specifically is it about the quality bonus payment system that incentivizes higher quality? For example, is it the amount of the payment? The measures the state is using? The ability to benchmark a clinic to others in the state?

  3. Have there been any surprises in how the quality bonus payment system has played out in your state or has it worked as expected? Any other reflections on quality bonus payments you’d like to share?

IV. Costs

A. Evaluation question: Did PPS rates change over time?

  1. Have your state’s CCBHCs encountered any difficulties reporting costs via the cost reports?

  2. Did the state rebase its PPS rates or adjust them for inflation in the last year?

    1. [If yes]: What led to the decision to do so? How did you use cost reports for rebasing?

  3. When did the state last rebase its PPS rates?

    1. When does the state plan to rebase its PPS rates next?

  4. Has the way the state approaches the rate setting process changed since we last spoke with you last year? How if so?

  5. Please provide the current PPS rates for each of your state’s CCBHCs (including any special population rates for PPS-2/4 states).

  6. [CARES Act states] We have seen that CCBHC costs in your state have [increased/decreased/stayed the same] over time. Do you have a sense for what could explain this?

  7. [CARES Act states, if applicable based on cost report analyses] Do you have a sense for what could explain variation in costs across CCBHCs? For example, was it primarily due to CCBHC location (urban vs. rural clinics), challenges hiring staff in some areas, or other reasons?

  8. Did you need to make new appropriations to fund the demonstration since we last spoke? How much is currently allocated for the demonstration?

B. Evaluation question: How did the PPS system support state and CCBHC efforts to improve access, quality of care, and scope of services?

  1. Have all services within the CCBHC scope of services historically been reimbursable by Medicaid in your state?

    1. [If no] Which services required by the CCBHC criteria have not historically been reimbursable?

  2. How is the PPS working for your state? How does it compare to the way your state reimburses other behavioral health providers?

  3. How did the PPS system support your state’s efforts to improve access, quality of care, and scope of services, if at all? What about for clinics?

    1. What do you think accounts for the PPS’ impacts? For example, are they from net increases in reimbursement, reductions in administrative burden, or coverage of previously uncovered services?

    2. Has the PPS facilitated any specific innovations? For example, innovative ways of service provision, uses of technology?

  4. Did the state expect to see savings from the demonstration?

  5. Have you been able to determine whether the state has seen any savings as a result of the demonstration?

    1. Were these savings experienced in the Medicaid program? Or other service systems (behavioral health, criminal justice, etc.)?

  6. If state Medicaid costs increased as a result of the demonstration, were those increases fully covered by the enhanced demonstration FMAP?

Year 4 (Fall 2026 - CARES Act Cohort in DY6, BSCA Cohort 1 in DY3, BSCA Cohort 2 in DY1)

I. Introduction

  1. We would like to learn about your current role(s)/position(s). What are your key responsibilities related to the CCBHC demonstration?

    1. For how long have you held this position?

  2. How has demonstration implementation been going for your state? What has been your greatest success and what has been your greatest challenge during the past year?

  3. Has the state’s role in demonstration implementation evolved since we last spoke with you? Please describe if so.

  4. [BSCA Cohort 2] When did the demonstration begin in [state]?

    1. Based on our latest records, it looks like there are currently [number] CCBHCs participating in the demonstration. Is that right?

    2. Do you have a sense for how many of the state’s demonstration CCBHCs were CCBHC-E grantees prior to the demonstration?

    3. Were CCBHCs funded by another funding mechanism prior to the demonstration, such as a state plan amendment or Section 1115 demonstration waiver?

  5. [BSCA Cohort 2] What initial challenges has your state agency experienced in the first few months of the demonstration?

  6. [BSCA Cohort 2] What initial challenges have demonstration CCBHCs in [state] experienced in the first few months of the demonstration?

  7. [BSCA Cohort 2] What types of services did CCBHCs in your state focus on providing prior to joining the demonstration? For example, did they primarily provide SUD or mental health services?

    1. What changes did CCBHCs have to make to be able to provide the full range of CCBHC services?

  8. [BSCA Cohort 2] Did state regulations or policies need to be altered to accommodate the CCBHC model? Please describe if so.

  9. [BSCA Cohort 2] Does your state plan to monitor CCBHCs’ adherence to the certification criteria in an ongoing way? How if so?

    1. Will clinics monitor adherence themselves?

    2. How does the state work with CCBHCs to ensure they continue to meet the criteria?

  10. What kinds of technical assistance has the state obtained or provided to CCBHCs recently?

II. Access

A. Evaluation question: What activities did CCBHCs implement to increase and maintain access to care? How do these activities compare with those of clinics in other regions of the state?

  1. [CARES Act Cohort; BSCA Cohort 1] Could you please update us on state and CCBHC activities to increase access to care since we last spoke with you in fall 2024?

    1. Has the state changed any requirements related to access since we last spoke?

    2. What challenges have demonstration CCBHCs encountered in increasing access to care? For example, CCBHCs may have encountered challenges in retaining staff to meet increased demands for services or encountered challenges engaging people in services.

  2. [CARES Act Cohort; BSCA Cohort 1] How does access to care in areas served by CCBHCs compare to areas not served by CCBHCs? What effect has the demonstration had on in areas CCBHC serve relative to other areas of the state?

    1. [If applicable] What do you think accounts for these differences?

  3. [CARES Act Cohort, BSCA Cohort 1] How have CCBHC efforts to increase access to care evolved over the course of the demonstration in your state?

    1. What kinds of refinements did CCBHCs make to their strategies or approach to increasing access to care in later years of the demonstration?

  4. [BSCA Cohort 2] How successful have the state’s CCBHCs been in meeting the demonstration’s access requirements so far? Have CCBHCs experienced any challenges meeting and of the access requirements?

    1. [If applicable] Why have these activities been challenging?

    2. Has the state offered any technical assistance to help CCBHCs meet the demonstration’s access requirements? What has this TA targeted if so?

  5. What specific elements of the CCBHC model and its requirements have most helped or hindered efforts to improve access to care under the demonstration? (E.g., the PPS, quality measures or bonus payments?)

III. Quality and Scope of Services

B. Evaluation question: Did CCBHCs establish relationships with DCOs or other providers? How do the approaches to establishing relationships of CCBHCs in the cohorts of states compare with those of the original states?

  1. To what extent are DCOs providing some of the required services in the state?

    1. If applicable:

    2. How many CCBHCs in the state are partnering with DCOs?

      1. Which services are DCOs providing? Why have CCBHCs elected not to provide these services directly?

    3. Why have some CCBHCs elected not to partner with DCOs?

    4. What steps does your state take to ensure that required CCBHC services provided by DCOs are delivered in a manner that meets the standards set in the CCBHC certification criteria?

    5. Does the state take any steps to monitor the quality of care provided at DCOs?

  2. Have CCBHCs in your state been successful at developing partnerships with other non-DCO external providers?

    1. Have CCBHCs been able to establish formal, signed care coordination agreements with external partners?

      1. Have certain types of external partners been more or less amenable to establishing formal, signed agreements than others (e.g., schools, VA facilities)? Why if so?

    2. What has facilitated and what has hindered development of partnerships with external partners?

    3. Has the state provided any support to CCBHCs in developing partnerships with external providers? What form has this support taken (e.g., technical assistance, learning collaboratives)?

  3. Could you reflect on the strength of relationships between CCBHCs and primary care providers in your state? Has the relationship between CCBHCs and primary care providers changed since the beginning of the demonstration?

  4. Could you reflect on the strength of relationships between CCBHCs and social service organizations in your state, such as housing or Medicaid agencies? Has the relationship between CCBHCs and social service organizations changed since the beginning of the demonstration?

  5. Is health IT supporting CCBHCs’ partnerships with other providers? What sorts of health IT are CCBHCs using to support partnerships with other providers if so?

    1. Do clinics have access to a health information exchange? How do they use this exchange to coordinate care? What are some of the challenges?

    2. Do clinics tend to receive alerts when a client is seen by another provider? Why or why not?

    3. How does technology support transitions of care for CCBHC clients in this state?

C. Evaluation question: How do states and CCBHCs collect, report, and use information to improve quality of care?

[BSCA Cohort 2]

  1. How are quality measures data going to be collected during the demonstration?

    1. What has the state done to help CCBHCs prepare to collect quality measures?

    2. What changes has the state made to its systems to prepare to collect quality measures?

  2. How will quality measures data be used during the demonstration (e.g., performance monitoring, continuous quality improvement efforts)?

  3. Will information on CCBHC quality measures be shared among various state agencies, with CCBHCs, consumers, families, and with the public?

  4. Prior to the demonstration, did your state require CMHCs or other behavioral health providers to report quality measures? For what purpose if so?

[CARES Act Cohort, BSCA Cohort 1]

  1. Could you reflect on your state’s experience collecting and reporting the CCBHC quality measures since we last spoke?

    1. What has hindered reporting?

    2. What has facilitated reporting?

    3. Has your state been able to calculate the measures following the specifications?

    4. Has quality measure reporting gotten easier over time? Why or why not?

    5. Do you feel like these were the right measures to monitor for the demonstration? Why or why not?

  2. Has the state used performance on quality measures to help clinics improve care recently? How if so?

  3. Does the state share performance on quality measures with CCBHCs?

    1. [If yes] In what time frame do you share these results?

    2. [If yes] What specifically does your state share? For example, do you share an individual clinic’s performance only or do you share performance of other CCBHCs in the state with clinics?

  4. Has the state used quality measure performance to change any policies? Please describe if so.

  5. Could you briefly share your reflections on how the quality of care delivered by CCBHCs compares to other behavioral health providers? How has the quality of care provided by CCBHCs changed [CARES Act Cohort: over the last few years / BSCA Cohort 1: since the beginning of the demonstration]?

[All cohorts]

  1. How is the state supporting CCBHCs’ continuous quality improvement activities?

IV. Costs

A. Evaluation question: How did states select PPS systems and set rates?

  1. Is it correct that your state opted for [PPS-1/2/3/4]? Why did your state select this option?

  2. [BSCA Cohort 2] How does the PPS for CCBHCs differ from existing funding mechanisms for behavioral health clinics in the state?

    1. Are there any other behavioral health clinics operating under similar systems, for example, through managed care arrangements?

  3. [BSCA Cohort 2] Are mental health or substance use disorder services covered by a Medicaid PPS-type/value-based bundled payment system in your state (other than the CCBHC PPS)?

    1. If yes, how does the CCBHC PPS compare with those systems?

  4. [BSCA Cohort 2] What data sources were used to calculate the CCBHC prospective payment rates for the first demonstration year for each clinic? (e.g., some states had CCBHCs complete hypothetical cost report)

    1. How were estimates generated for the full scope of services included in the CCBHC rate?

    2. Did the state engage an outside contractor to assist with rate setting?

    3. How did the calculation of the proposed PPS rate take into account the rates for services prior to the demonstration?

    4. Are there services that were formerly not reimbursed by Medicaid, but are under the CCBHC demonstration? How did the calculation of the proposed PPS rate take these into account if so?

  5. Please provide the current PPS rates for each of your state’s CCBHCs (including any special population rates for PPS-2/4 states).

  6. Has the state made any changes to its billing processes or infrastructure to support the demonstration? (For example, some states significantly updated billing IT systems to streamline payments to CCBHCs)

    1. Did the state or contractors provide any technical assistance to help CCBHCs with billing process improvements?

V. Cross-cutting Topics

A. Evaluation question: How did states and CCBHCs use findings from their community needs assessments to inform demonstration implementation?

  1. [CARES Act Cohort, BSCA Cohort 1] To confirm, clinics complete their own community needs assessments, correct? How often do the needs assessments happen in your state? When did clinics last conduct a community needs assessments?

    1. [If recent]: How does the state use clinics’ community needs assessments? What did you learn from them?

      1. Is the state making any changes in response to the findings?

      2. How did clinics involve people who receive services in the needs assessments?

    2. [If hasn’t happened recently]: When will your community needs assessments next take place? In general, how does the state use/plan to use clinics’ community needs assessments? How does the state (plan to) include the perspectives of people receiving services?

  2. [BSCA Cohort 2] How did the state use findings from CCBHCs’ community needs assessments to inform demonstration requirements?

  3. [BSCA Cohort 2] How frequently will CCBHCs be required to complete community needs assessments during the demonstration? Are you requiring them more frequently than the criteria indicate (at least every 3 years)?

C. Are states planning to sustain the CCBHC model through other policy initiatives (for example, 1115 waivers, state plan amendments)? Do they plan to make changes to state CCBHC requirements or adapt implementation in any way (for example, modifying required services or quality measures).

  1. [CARES Act Cohort] Is your state planning to sustain the model after the demonstration ends? Why or why not?

    1. [If yes] How are you planning to sustain the model? (For example, section 1115 demonstration, state plan amendment)

    2. [If yes] Are you planning to make any changes to the requirements for CCBHCs?

      1. Will you retain all the required services and types of staff? Why?

      2. Will you require the same or different quality measures?

    3. [If yes] What specific funding mechanisms do you plan to use? Are you planning to make any changes to the payment system? Please describe.

Year 5 (Fall 2027 – BSCA Cohort 1 in DY4, BSCA Cohort 2 in DY2)

I. Introduction

  1. We would like to learn about your current role(s)/position(s). What are your key responsibilities related to the CCBHC demonstration?

    1. For how long have you held this position?

  2. How has demonstration implementation been going for your state? What has been your greatest success and what has been your greatest challenge during the past year?

  3. Has the state’s role in demonstration implementation evolved since we last spoke with you? Please describe if so.

  4. In general, have CCBHCs in your state been able to maintain the certification criteria since we spoke last year?

  5. Overall, how do you think the CCBHC demonstration implementation has gone in your state? Probe for: Differences in implementation across CCBHCs within the state (e.g., urban vs. rural, type of clinic prior to CCBHC certification, populations served, etc.)

    1. How did your experience compare with your expectations for the demonstration?

  6. What are some of the key successes the demonstration has had?

    1. What factors have played a role in demonstration successes?

  7. [BSCA Cohort 2] What problems or barriers have CCBHCs in your state faced since the demonstration began?

    1. Were these barriers anticipated or unexpected?

    2. What steps have been taken to address or resolve these barriers? Have these actions been effective?

  8. How have people receiving CCBHC services, family members, providers, and other stakeholders been involved in the demonstration since it began?

    1. What critical issues have they raised?

    2. How has their input influenced the demonstration in your state?

II. Access

A. Evaluation question: Did the number and characteristics of clients served by CCBHCs change across demonstration years?

  1. Did the state serve the number of people it originally projected serving when it applied for the demonstration? Why or why not?

    1. Were there any surprises in the characteristics of people served?

    2. [BSCA Cohort 2 (QM caseloads will not be available)] Have the characteristics of people served stayed the same over time?

    3. [If yes] What has helped CCBHCs serve the number and types of people expected?

  2. Could you please reflect on how the CCBHC model affects use of behavioral health services? What about use of physical health care?

    1. Have utilization patterns changed since the demonstration was implemented?

      1. Do you think people are using behavioral and physical health services more or less than or in a different way than they would have if they were receiving care from a non-CCBHC behavioral health organization? Why?

      2. Have you assessed this quantitatively? For example, have you collected data that show changes in utilization patterns?

    2. Have referral patterns changed since the demonstration was implemented?

B. Evaluation question: What successes and challenges did new states and CCBHCs encounter in improving access to care? How did they overcome challenges? [BSCA Cohort 2 ONLY]

  1. Could you please update us on state and CCBHC activities to increase access to care since we last spoke with you in fall 2026?

    1. What kinds of technical assistance or guidance have the state or other groups provided this past year to support CCBHCs’ efforts to increase access (if any)?

    2. Has the state changed any requirements related to access since we last spoke?

  2. Could you please share some of the key successes that demonstration CCBHCs have had in increasing access to care?

  3. What challenges have demonstration CCBHCs encountered in increasing access to care? [Potential probes]:

    1. Can you think of any statewide or regional challenges? What about challenges for specific clinics?

    2. Do you recall any activities that CCBHCs considered, but did not end up implementing due to challenges?

    3. Have CCBHCs discontinued any activities originally implemented to increase access? Why?

    4. Have there been any access challenges specific to certain populations? (For example, consumers with disabilities, homeless, children, adolescents, Veterans, Indigenous people, rural)

    5. How, if at all, is the state supporting the CCBHC(s) to address these issues?

    6. How are CCBHCs working to overcome these challenges? What have been some of the more successful strategies to date?

II. Quality and Scope of Services

A. Evaluation question: How did the scope of services provided by CCBHCs compare with those of clinics in the same regions or other regions of the state?

  1. How does the scope of services provided by CCBHCs in your state compare with non-CCBHC behavioral health clinics in the same regions that are served by CCBHCs? What about those in other regions of the state? Do other non-CCBHC behavioral health organizations in the state provide this full scope of services?

    1. How does this vary, if at all, by geographic area? (e.g., rural, urban, frontier)

    2. Do people served by CCBHCs continue to receive services from other behavioral health providers in the community? Why?

B. Evaluation question: Did CCBHCs maintain the required services over time?

  1. Have there been any major changes in the scope of services provided by CCBHCs in the last year or two? For example, have CCBHCs struggled to maintain any of the required services? Please describe if so.

  2. Has the state provided any new guidance or support or made any new investments to help CCBHCs provide the required services since we last spoke with you?

    1. [If yes]: What prompted those actions?

  3. What are some barriers that clinics in your state have faced in providing the full CCBHC scope of services over the course of the demonstration?

  4. Have CCBHCs in your state been able to maintain the required CCBHC services over time?

    1. Have some services been more difficult to maintain than others? Why?

    2. What has helped CCBHCs maintain the required services?

C. Evaluation question: What challenges did CCBHCs encounter in providing particular types of services or providing services to certain target populations?

  1. Have CCBHCs in your state encountered challenges in providing any of the required types of services?

    1. If yes, which services have been challenging? Have these challenges changed over time?

  2. Have CCBHCs in your state encountered any challenges in providing services to certain target populations?

    1. What populations and services were challenging to serve?

    2. Were challenges specific to certain areas of the state or CCBHCs?

    3. Did the state provide any support or guidance to help CCBHCs serve different target populations?

  3. [BSCA Cohort 2] How are case management and care coordination activities going for CCBHCs?

    1. What are some early challenges and successes of this work?

    2. Are the care coordination services that CCBHCs offer substantially different from those available from other community behavioral health organizations in your state?

    3. Have CCBHCs generally been able to maintain care coordination activities during the demonstration?

    4. What investments, support, or guidance has the state provided to help CCBHCs coordinate care externally?

    5. How do CCBHCs learn of care transitions among people served by the CCBHC? For example, hospital stays or ED visits?

  4. Has the state implemented any new requirements related to staffing in the past year?

    1. Why did the state make these changes?

  5. [BSCA Cohort 2] Have your state’s CCBHCs been successful in hiring the numbers and types of staff reflected in their staffing plans?

    1. Did CCBHCs’ community needs assessments identify any staffing needs?

    2. Have certain types of staff been more challenging to hire than others? Has this changed since the beginning of the demonstration?

    3. What factors have influenced CCBHCs’ ability to hire required staff?

    4. Has the state done anything to assist CCBHCs in hiring needed staff or addressing staff hiring challenges?

  6. [BCSA Cohort 1] Have your state’s CCBHCs been successful in retaining staff needed to implement the CCBHC model during the demonstration?

    1. [If yes]: What has helped clinics to retain staff?

    2. [If no]: What challenges have clinics faced in retaining the required staff? For example, have certain staff types been more difficult to retain than others?

      1. What caused these challenges?

      2. Has the state done anything to assist CCBHCs in addressing these challenges?

    3. Has clinics’ ability to retain staff differed at different points during the demonstration? What accounts for this variation if so?

    4. Have certain CCBHCs had more trouble retaining staff than others? What accounts for this difference?

D. Evaluation question: Did CCBHCs establish relationships with DCOs or other providers? How do the approaches to establishing relationships of CCBHCs in the cohorts of states compare with those of the original states? [BSCA Cohort 2 ONLY]

  1. To what extent are DCOs providing some of the required services in the state?

    1. If applicable:

    2. How many CCBHCs in the state are partnering with DCOs?

      1. Which services are DCOs providing?

      2. Why have CCBHCs elected not to provide certain services directly?

    3. Why have some CCBHCs elected not to partner with DCOs?

    4. What steps does your state take to ensure that required CCBHC services provided by DCOs are delivered in a manner that meets the standards set in the CCBHC certification criteria?

    5. Does the state take any steps to monitor the quality of care provided through DCOs? How do CCBHCs ensure that DCOs provide care consistent with the certification criteria?

  2. Have CCBHCs in your state been successful at developing care coordination partnerships with other non-DCO external providers?

    1. Have CCBHCs been able to establish formal, signed care coordination agreements with external partners?

      1. Have certain types of external partners been more or less amenable to establishing formal, signed agreements than others (e.g., schools, VA facilities)? Why if so?

        • In cases where partners are not open to formal, signed agreements, have CCBHCs been able to develop unsigned joint protocols with them instead? What do partnerships look like if not?

    2. What has facilitated and what has hindered development of care coordination partnerships with external partners?

    3. Has the state provided any support to CCBHCs in developing care coordination partnerships with external providers? What form has this support taken (e.g., technical assistance, learning collaboratives)?

  3. Could you reflect on the strength of relationships between CCBHCs and primary care providers in your state? Has the relationship between CCBHCs and primary care providers changed since the beginning of the demonstration?

  4. Could you reflect on the strength of relationships between CCBHCs and social service organizations in your state, such as housing or Medicaid agencies? Has the relationship between CCBHCs and social service organizations changed since the beginning of the demonstration?

  5. Is health IT supporting CCBHCs’ partnerships with other providers? What sorts of health IT are CCBHCs using to support partnerships with other providers if so?

    1. Do clinics have access to a health information exchange? How do they use this exchange to coordinate care? What are some of the challenges?

    2. Do clinics tend to receive alerts when a client is seen by another provider? Why or why not?

    3. How does technology support transitions of care for CCBHC clients in this state?

E. Evaluation question: What was the quality of care provided to CCBHC clients? How did it compare to state benchmarks? Did quality of care change over time?

  1. Could you reflect on your state’s experience collecting and reporting the CCBHC quality measures since we last spoke?

    1. What has hindered collecting and reporting?

    2. What has facilitated collecting and reporting?

    3. Have you been able to calculate the measures following the specifications?

  2. [BSCA Cohort 1: Are the required quality measures /BSCA Cohort 2: Do you expect the required quality measures will be] helpful for measuring and improving the quality of your CCBHC and the quality of the care that your CCBHC provides?

    1. Why or why not?

    2. Does your CCBHC collect or plan to collect and use the optional measures? What about additional measures not included as required or optional for CCBHCs? Why or why not?

  3. [BSCA Cohort 1] Has the state seen marked improvement or worsening of any of the quality measures?

    1. Which measure(s)? [Interviewer note: have a list of quality measures available for reference]

    2. What do you think accounts for this change?

  4. Reflecting on the CCBHC model overall, do you think it improves the quality of behavioral health care (compared to standard practice provided by other behavioral health providers in your state)?

    1. If yes: Which aspects of the model improve care quality the most? For example, have any specific evidence-based practices influenced the quality of care?

    2. How has the quality of care provided by CCBHCs changed over time?

  5. Are there any CCBHC requirements that do not significantly improve the quality of care? Why? (For example, because they are too burdensome, not well-received, etc.)

  6. How, if at all, is the state supporting CCBHCs’ continuous quality improvement activities?

    1. How has the state used performance on quality measures to help clinics improve quality, if at all?

    2. Have the state or other groups offered any technical assistance related to quality?

  7. Has the state used quality measure performance to change any policies? Please describe if so.

III. Costs

A. Evaluation question: How did the PPS system support state and CCBHC efforts to improve access, quality of care, and scope of services?

  1. Has the state made any changes to its PPS or the way the state approaches the rate setting process in the last year?

  2. Did CCBHCs encounter any difficulties with respect to cost reporting? Please describe.

  3. Please provide the current PPS rates for each of your state’s CCBHCs (including any special population rates for PPS-2/4 states).

  4. Did the state rebase its PPS rates or adjust them for inflation in the last year?

    1. [If yes]: What led to the decision to do so? How did you use cost reports for rebasing?

  5. When did the state last rebase its PPS rates?

    1. When does the state plan to rebase its PPS rates next?

  6. How is the PPS working for your state?

    1. [BSCA Cohort 2] How does it compare to the way your state reimburses other behavioral health providers?

  7. How did the PPS system support your state’s efforts to improve access, quality of care, and scope of services, if at all? What about for clinics?

    1. What do you think accounts for the PPS’ impacts? For example, are they from net increases in reimbursement, reductions in administrative burden, or coverage of previously uncovered services?

    2. Has the PPS facilitated any specific innovations? For example, innovative ways of service provision, uses of technology?

  8. Did the state expect to see any savings as a result of the demonstration?

  9. Have you been able to determine whether the state has seen any savings as a result of the demonstration?

    1. Were these savings experienced in the Medicaid program? Or other service systems (behavioral health, criminal justice, etc.)?

  10. If state Medicaid costs increased as a result of the demonstration, were those increases fully covered by the enhanced demonstration FMAP?

  11. Did you need to make new appropriations to fund the demonstration since we last spoke? How much is currently allocated for the demonstration?

B. Evaluation question: How did states structure their quality bonus payment systems? Did state awards of quality bonus payments change over time?

  1. Have there been any changes to the way the state awards/plans to award quality bonus payments since the state applied for the demonstration? For example, have there been any changes to measures or measure thresholds used or the extent reporting or improving on a measure factors into your approach?

  2. Have there been any surprises in how the quality bonus payment system has played out in your state or has it worked as expected? Any other reflections on quality bonus payments you’d like to share?

C. Evaluation question: Did the introduction of the CCBHC model impact hospitalization rates, ED visits, and ambulatory care visits for CCBHC clients relative to within-state comparison groups?

  1. Do you have a sense for whether the introduction of the CCBHC model has impacted use of specific types of care by people receiving services from CCBHCs? Let’s talk about different kinds of service use, starting with:

  • hospitalization rates

  • emergency department visits

  • ambulatory care visits



    1. If yes, is your perception anecdotal, or have you collected any data or conducted analyses on such impacts?

      1. [If based on data] Have you developed any written products based on those analyses that you could share with us?

  1. If the demonstration has had any impacts on service use, what components of the model do you think would have led to those impacts?

  2. Do you have a sense for how service utilization among people who receive services at CCBHCs compares with use by people not served by CCBHCs?

    1. If you perceive differences, what do you think accounts for these differences?

IV. Cross-cutting Topics

A. Evaluation question: Are states planning to sustain the CCBHC model through other policy initiatives (for example, 1115 waivers, state plan amendments)? Do they plan to make changes to state CCBHC requirements or adapt implementation in any way (for example, modifying required services or quality measures).

  1. Is your state planning to sustain the model after the demonstration ends? Why or why not?

    1. [If yes] How are you planning to sustain the model? (E.g, section 1115 demonstration, state plan amendment)

    2. [If yes] Are you planning to make any changes to the requirements for CCBHCs?

      1. Will you retain all the required services and types of staff? Why?

      2. Will you require the same or different quality measures?

    3. [If yes] What specific funding mechanisms do you plan to use? Are you planning to make any changes to the payment system? Please describe.



Mathematica® Inc. 18

File Typeapplication/vnd.openxmlformats-officedocument.wordprocessingml.document
File Title1-column report template
AuthorAllison Wishon
File Modified0000-00-00
File Created2024-12-10

© 2025 OMB.report | Privacy Policy