Form 0920-25BW CDC eICR Data Project - Discussion Guide

CDC eICR Data Project: Discussion Guide

These are the interview guides that we are following to help extract specific information from participants during our conversations with staff from DCHHS and CDPH.

Technical SME interview guidance

Workflows

1. Facilitator leads the discussion (~45 mins)

2. SMEs ask technical follow-up questions (~15 mins)

Systems, Tech, Policy

1. Facilitator leads the intros, context setting & overall discussion (~15 mins)

2. Facilitator leads review of system diagram & SMEs ask technical questions (~45 mins)

Workflows

Introduction

Introduce yourself, set expectations, and re-confirm consent to record (if applicable).

Hi! Thank you so much for joining us today. My name is [name] and I’m a design strategist at Skylight. We’re a digital consulting firm working with the CDC on a project related to eCR data. I also have a few teammates on the call—[names/roles]—who are helping with this discussion.. Before we begin, I want to provide you with some context about our session, and what to expect.

Our goal is to learn more about how you and your team work with eCRs—how the data moves through your jurisdiction, and what you’re doing with that info or how you’d like to leverage it in the future. Our intention is to use these learnings to improve the quality of the data you are receiving, for example by offering deduplication tools to make the data easier to use. So, I’ll be asking you questions related to your work in this area and the processes, people, and systems involved.

For the most part I’ll be guiding the conversation, and will leave some time toward the end of the call for my colleagues to ask any follow-up questions, and for you to share anything else you think is important to know about this topic that we didn’t cover.

Your participation is completely voluntary, and if at any point there are questions you don’t feel comfortable answering, just let me know and we can move on to another topic.

Do you have any questions before we begin?

[Skip if they declined to be recorded in Calendly]

As mentioned previously, we would like to record this session. Is that ok with you? This is just to supplement my note-taking and ensure that I can actively listen to you. We won’t record your identity or other sensitive information about you in our notes or documents.

[Press Record > Record to the cloud, or shortcut ⌘⇧C]

Background

Build rapport and get the participant talking freely.

• To get started, can you tell me a little about your role and what your job involves?

  • [Optional] How long have you been in your current role?

  • [Optional] What does a typical workday look like for you?

• Based on my notes, your responsibilities include … [list responsibilities they selected] …

  • Case investigation and follow-up: Use eCR data to initiate case investigations, contact patients or healthcare providers for additional information, and provide guidance on disease control measures.

  • Prepare eCR data for analysis: Cleaning, standardizing, and preparing the raw eICR data for epidemiological analysis and public health action.

  • Data analysis & reporting: Use eCR data to conduct statistical analyses, generate reports on disease trends and patterns, identify potential public health threats, and inform public health decision making.

  • System implementation & maintenance: Manage or implement data systems that relate to the flow of eICR data (e.g., Salesforce, Rhapsody, Mirth, Informatica, containerization services, data lakes, data warehouses), ensuring proper functioning, and troubleshooting any technical issues that arise.

  • Data integration: Integrate eCR data with other public health data sources, such as laboratory reports or immunization registries, to gain a more comprehensive understanding of disease patterns and risk factors.

  • eCR quality & improvement: Review the overall performance of the eCR system, assess the effectiveness of public health interventions, or identify areas for improvement in disease reporting and response.

  • Data governance & compliance: Apply expertise in data privacy, security, and compliance to develop and implement policies, procedures, or technical safeguards that ensure the compliant, secure, and effective use and sharing of eICR data.

• Which of those responsibilities …

  • Is the most painful or time-consuming?

  • Can you speak to in the most detail?

  • Relates to a recent experience you can talk through in detail?

Jobs

To get participants to talk about their jobs to be done.

• Quick Guide: Tell me about your goals with that. What problems are you trying to prevent or solve?

• As far as your responsibilities go in relation to case reporting, what are your main goals? What problems are you trying to prevent or resolve?

• Is duplicate data from eICRs a common problem for your team? How severe is it, and what methods do you use to identify and address it?

• [If yes] When managing duplicate eICRs, are you mainly looking to remove redundant information within or across documents, or are you trying to determine if the eICR belongs to a patient already in your system?

• What other manual work gets in the way of your goals for eICR & case reporting?

• Which tools, people, or processes help you achieve your goals in case reporting?

• Which responsibilities in the case reporting process would you solve with a magic wand? What would that solution look like?

Process

Go through the stages of getting the job done.

• Quick Guide: Tell me about the steps involved.

• Can you think back to the last time you worked on setting up eCRs for case reporting? Can you go step by step of how you did that?

• For each step, could you also tell me how you make decisions along the way?

• How do you know when you're done and it's done right? Is there a feedback loop?

Needs

Uncover the desired outcomes that people are looking for while performing the job.

• Quick Guide: Tell me about any workarounds you use for painful parts of this process.

• What workarounds have you used when supporting eCR and case reporting? How much time do you spend on those and other workarounds?

• What are the most painful problems or blockers you encounter in your case reporting work?

• What could be easier? Why?

• How do you feel when the job is completed?

Circumstance

Find out when and where performing the job makes a difference.

• Quick Guide: Tell me about the environment that this work happens in (physical, digital, cultural, time based, etc.)

• What circumstances led up to the last time you worked to support your team with eCR and case reporting?

• What influences the choices you're making during this process?

• What outside factors (systems, process, people, environment, etc.) would cause you to do this process differently?

• What is motivational for getting this process done? What's demotivating?

• Are there any milestones that lead up to or follow this process?

Wrap-up

Allow the participant time to share their feedback or questions, and thank them for their time.

That’s all the questions we have. Before we wrap up…

• Is there anything else you would like to share that we haven't covered?

• Do you have any questions for us?

Thanks again for your time and participation in this interview. Your perspective has been incredibly valuable, and they will be a big help in informing our work. If you have any additional questions or concerns, please don't hesitate to reach out.

(DALLAS) Systems, Tech, Policy

Introduction

Introduce yourself, set expectations, and re-confirm consent to record (if applicable).

Hi! Thank you so much for joining us today. My name is [name] and I’m a design strategist at Skylight. I also have a few teammates on the call—[names/roles]—who are helping with this discussion. Before we begin, I want to provide you with some context about our session, and what to expect.

The purpose of this session is for us to learn about how you and your team work with eICRs and case reporting (i.e., eCR ingestion and utilization for case investigation). Our intention is to use these learnings to improve the quality of the data you are receiving, for example by offering deduplication tools to make the data easier to use. I will start off the conversation with some questions related to your work in this area and the processes, people, and systems involved. Then, I will hand it over to my colleague [name], who will share a system diagram and ask some questions about [jurisdictions]’s technical setup for processing eCR data.

We will leave some time toward the end of the call for my colleagues to ask any follow-up questions, and for you to share anything else you think is important to know about this topic that we didn’t cover.

If at any point there are questions you don’t feel comfortable answering, just let me know and we can move on to another topic.

[Skip if they declined to be recorded in Calendly]

As mentioned previously, we would like to record this session. Is that ok with you? This is just to supplement my note-taking and ensure that I can actively listen to you. We won’t record your identity or other sensitive information about you in our notes.

[Press Record > Record to the cloud, or shortcut ⌘⇧C]

Background

Build rapport and get the participant talking freely.

• Our goal today is to learn from you about how your team handles eICRs and case reporting (i.e. eCR ingestion and utilization for case investigation). So, we'd like to know about systems, people, processes and anything else you'd like to share.

• Based on my notes, your responsibilities include … [list responsibilities they selected]

• Given your background, we're hoping you can answer a handful of technical questions.

Jobs

To get participants to talk about their jobs to be done.

• Quick Guide: Tell me about your goals with that. What problems are you trying to prevent or solve?

• Before we dive into the weeds, let's talk about goals. When you're setting up and maintaining these systems for eCR and case reporting, what are your main goals? What problems are you trying to prevent or resolve?

Needs

Uncover the desired outcomes that people are looking for while performing the job.

• Quick Guide: Tell me about any workarounds you use for painful parts of this process.

• Tell me about any workarounds you use for painful parts of setting up or maintaining these systems.

Circumstance

Find out when and where performing the job makes a difference.

• Quick Guide: Tell me about the environment that this work happens in (physical, digital, cultural, time based, etc.)

• What influences the choices you're making while setting up or maintaining these systems?

Transition To Diagram

Lead into the technical discussion

• Remember: Ask "why" to zoom out & "how" to zoom in.

• We found a diagram online that we think represents your full data pipeline journey - first, is this accurate (share screen)?

Data Source Systems

• For all tools mentioned: Why was this software tool chosen?

• What are the specific data types and formats being exchanged?

• How do eCRs differ across various EHR systems you interact with?

• What obstacles did you run into when initially implementing this system for COVID-19 (funding, policy, etc.)?

• What obstacles (policy, security, prototypes, etc.) did you run into when you started ingesting eICRs from other conditions besides COVID?

• How is eICR data transformed throughout this process, and what challenges arise when preparing the data for use?

• Where do you and your team feel limited or burdened due to maintenance of these systems, whether due to lack of time, personnel, or other resources?

SFTP ( Routing)

• For all tools mentioned: Why was this software tool chosen?

• What does this workflow/pipeline look like from a technical perspective? Are there standard requirements for new and existing data sources to integrate this protocol with current Informatica MDM?

Data Engine

• For all tools mentioned: Why was this software tool chosen?

• What manual work gets in the way of your goals for this particular area of focus?

• Let's talk about duplicate data from eICRs. Is this a common problem for your team (est. of % of incoming eICR)? How severe is it, and how do you identify and address it (est. of how long it takes to address)? What would you do with the time you gained if you didn’t have to do this?

• [If yes] Is addressing duplicate data an automated process? If so, what gaps or pain points exist in that solution?

• [If yes] How do you balance the access to full medical history with reducing redundant data?

• What solutions to address data duplication are currently in place, and how effective are they?

  • Why was Informatica MDM selected for data standardization, matching, deduplication, and cleansing? Is the plan moving forward to keep using it or are there intentions to move away from it?

• How is Informatica MDM and the Database Warehouse currently hosted?

• How has performance or user experience with the current system been measured?

• Which hosting solution—centrally hosted or self-hosted—better aligns with your capabilities and needs? What would you consider as the essential components for centralized hosting? (ex. authentication, connectivity, storage, data privacy)?

Target System & Reporting (Surveillance)

• For all tools mentioned: Why was this software tool chosen?

• What types of analysis do epidemiologists wish they could perform with eCR data?

• Why was Salesforce selected as a surveillance system, and what are its advantages and disadvantages?

• What Salesforce products are being used and how is Salesforce currently hosted?

• What are the long-term plans for implementing Salesforce as a surveillance system?

• Do you have any data privacy concerns using your system?

• If you address duplicate data in an automated way, how does it perform in terms of false positives, false negatives, and processing time?

• How, if at all, does the system allow for humans in the loop to manually review processed eICRs for quality control and data integrity?

• What is the current use case and solution for viewing eCR/eICR data? What are the likes and dislikes with the current system?

Wrap-up

Allow the participant time to share their feedback or questions, and thank them for their time.

That’s all the questions we have. Before we wrap up…

• Is there anything else you would like to share that we haven't covered?

• Do you have any questions for us?

Thanks again for your time and participation in this interview. Your perspective has been incredibly valuable, and they will be a big help in informing our work. If you have any additional questions or concerns, please don't hesitate to reach out.

(CHICAGO) Systems, Tech, Policy

Introduction

Introduce yourself, set expectations, and re-confirm consent to record (if applicable).

Hi! Thank you so much for joining us today. My name is [name] and I’m a design strategist at Skylight. I also have a few teammates on the call—[names/roles]—who are helping with this discussion. Before we begin, I want to provide you with some context about our session, and what to expect.

The purpose of this session is for us to learn about how you and your team work with eICRs and case reporting (i.e., eCR ingestion and utilization for case investigation). Our intention is to use these learnings to improve the quality of the data you are receiving, for example by offering deduplication tools to make the data easier to use. I will start off the conversation with some questions related to your work in this area and the processes, people, and systems involved. Then, I will hand it over to my colleague [name], who will share a system diagram and ask some questions about [jurisdictions]’s technical setup for processing eCR data.

We will leave some time toward the end of the call for my colleagues to ask any follow-up questions, and for you to share anything else you think is important to know about this topic that we didn’t cover.

If at any point there are questions you don’t feel comfortable answering, just let me know and we can move on to another topic.

[Skip if they declined to be recorded in Calendly]

As mentioned previously, we would like to record this session. Is that ok with you? This is just to supplement my note-taking and ensure that I can actively listen to you. We won’t record your identity or other sensitive information about you in our notes.

[Press Record > Record to the cloud, or shortcut ⌘⇧C]

Background

Build rapport and get the participant talking freely.

• Our goal today is to learn from you about how your team handles eICRs and case reporting (i.e. eCR ingestion and utilization for case investigation). So, we'd like to know about systems, people, processes and anything else you'd like to share.

• Based on my notes, your responsibilities include … [list responsibilities they selected]

• Given your background, we're hoping you can answer a handful of technical questions.

Jobs

To get participants to talk about their jobs to be done.

• Quick Guide: Tell me about your goals with that. What problems are you trying to prevent or solve?

• Before we dive into the weeds, let's talk about goals. When you're setting up and maintaining these systems for eCR and case reporting, what are your main goals? What problems are you trying to prevent or resolve?

Needs

Uncover the desired outcomes that people are looking for while performing the job.

• Quick Guide: Tell me about any workarounds you use for painful parts of this process.

• Tell me about any workarounds you use for painful parts of setting up or maintaining these systems.

Circumstance

Find out when and where performing the job makes a difference.

• Quick Guide: Tell me about the environment that this work happens in (physical, digital, cultural, time based, etc.)

• What influences the choices you're making while setting up or maintaining these systems?

Transition To Diagram

Lead into the technical discussion

• Remember: Ask "why" to zoom out & "how" to zoom in.

• We found a diagram online that we think represents your full data pipeline journey - first, is this accurate (share screen)?

Source

• For all tools mentioned: Why was this software tool chosen?

• How do eCRs differ across various EHR systems you interact with?

• What obstacles did you run into when initially implementing this system (funding, policy, etc.)

• What conditions are you ingesting eICRs for and if it's more than 1 condition, what obstacles (policy, security, etc.) did you run into as you expanded into other conditions?

• How is eICR data transformed throughout this process, and what challenges arise when preparing the data for use?

• How do you utilize the CCD data that you receive? And do the same challenges that exist for eICR data exist for CCD data?

• What data do you receive from the CDC, if any?

• Where do you and your team feel limited or burdened due to maintenance, whether due to lack of time, personnel, or other resources?

Mechanism

• For all tools mentioned: Why was this software tool chosen?

• What does this workflow/pipeline look like from a technical perspective? Are there standard requirements for new and existing data sources to integrate this protocol?

Processing

• For all tools mentioned: Why was this software tool chosen?

• What manual work gets in the way of your goals for this particular area of focus?

• Let's talk about duplicate data from eICRs. Is this a common problem for your team? How severe is it, and how do you identify and address it?

• [If yes] Is addressing duplicate data an automated process? If so, what gaps or pain points exist in that solution?

  • If so, what gaps or pain points exist in that solution?

  • If not, why haven’t you explored automating the process? Due to resource constraints, or is there information you have relied on humans to detect in the process thus far?

• [If yes] How do you balance the access to full medical history with reducing redundant data?

• What solutions to address data duplication are currently in place, and how effective are they?

• Why was Azure selected as the cloud provider, along with associated products (Azure Blob Storage, Azure Automation, Azure Data Factory)?

• Why was Databricks selected as a tool for data processing and how does your team use Databricks?

• How has performance or user experience with the current system been measured?

Storage

• For all tools mentioned: Why was this software tool chosen?

• In terms of data processing tools, which hosting solution—centrally-hosted or self-hosted—better aligns with your capabilities and needs? What would you consider as the essential components for a centrally-hosted data processing solution? (ex. authentication, connectivity, data security & privacy, compliance)

Surveillance

Chicago does not have/use a surveillance system as a department.

The Syndemic Infectious Disease Bureau (SIDB) has an instance of Maven, called CHIMS, which they use for surveillance and disease intervention/investigation activities. The work to integrate eCR to the SIDB workflow is just beginning now, so that part is a question mark for now.

The Communicable Disease (CD) Bureau’s surveillance work and investigations happen across a few platforms. They use REDCap projects for some diseases, and those not captured in REDCap, are documented in INEDSS. The Illinois Dept. of Public Health, state surveillance system.

The Web App, a website- to view the eICR html’s is the current solution to use this pipeline, as a reference to gap fill provider and lab reports received from the state case report database (INEDSS).

• For all tools mentioned: Why was this software tool chosen?

• Why does the Chicago Department of Public Health not use one department-wide surveillance system?

• What are the pros (e.g. disease teams might not have access to irrelevant information thus enforcing a least-privilege approach to accessing data) and cons (e.g. redundant data, increased storage costs, etc) of not sharing a surveillance system

Wrap-up

Allow the participant time to share their feedback or questions, and thank them for their time.

That’s all the questions we have. Before we wrap up…

• Is there anything else you would like to share that we haven't covered?

• Do you have any questions for us?

Thanks again for your time and participation in this interview. Your perspective has been incredibly valuable, and they will be a big help in informing our work. If you have any additional questions or concerns, please don't hesitate to reach out.

DW2 (Hybrid Workflows / Systems Interview)

Background/Goals

• My understanding is that you’re the lead for all of the informatics-related projects at Dallas County and your primary role is building out this new surveillance system and dealing with our ELR feeds. Is that all correct? Any additions?

• As far as your work in building out your surveillance system, what are your main goals? What problems are you trying to prevent or resolve?

• How do you define and measure success?

• What’s the long-term vision for this system? Are you hoping to eventually use Salesforce for all conditions, or do you think NEDSS will still play a part in some conditions?

• What have been some of the big milestones on this project? What is the next big milestone you’re working toward?

• What are some of the big lessons learned on this project so far? What advice would you give to other jurisdictions who are interested in implementing a similar system?

• What challenges have you experienced in the process of scaling this system to new conditions? What are the biggest pain points?

  • Do you have any concerns about storage as you expand your system to accommodate more conditions?

Processes

• What steps are involved in the process of changing a condition from NEDSS to Salesforce for surveillance? Can you walk me through a recent example?

  • How does training work? Who’s responsible for training?

• You had previously talked about how with COVID, you’re able to automatically create investigations in Salesforce based on positive test results. For more complex conditions, you mentioned working on ways to use diagnostic codes or condition codes from combined eCR and RR files to automatically create cases in Salesforce. How is that process going? What challenges have you run into?

• It sounds like you’ve made a lot of progress with streamlining a lot of repetitive parts of the process. I’m curious to hear more about where you’ve run into limits of what can or should be automated, versus what tasks should still include some human intervention or review as part of the process?

• One thing that came up in another conversation was the fact that COVID is no longer a reportable condition in the state of Texas. Is that right? What effects has that had on your system, and has that changed how you approach taking in that information or storing historical data?

• How do you review data quality and the overall performance of the system? What problems come up? How do you resolve them?

Benefits

• What are some of the main benefits you receive from Salesforce, especially when compared to how things are done in NEDSS?

  • When it comes to… Case investigation

  • When it comes to… Reporting

• One of the benefits of Salesforce you mentioned in a previous interview was speed. Could you talk more about what this means? (Performance, fewer clicks?)

• Another benefit you mentioned was that capacity wise it's able to hold all of the data. Is that still the case?

• What were some of the things taking up the most time before?

Systems/Tech

• How is Salesforce hosted?

• What are some examples of how ELR and eCR data come together in the system? Are there any issues that come up?