Form approved
OMB # 0920-1050
Exp. date 06/30/2025
Attachment B – Usability Testing Script for CCSM CDS Interface
Usability test volunteers will be directed to a webpage where they can view sample test patients and launch the Cervical Cancer Screening and Management Clinical Decision Support (CCSM CDS) Dashboard interface.
Interviews will be conducted over Microsoft Teams, and recorded to observe and record user behavior, interactions, and record feedback for purposes of improving the dashboard.
Usability testing will be conducted by a lead evaluator, and another usability team member will be available to answer technical questions regarding the CCSM CDS operation.
The lead evaluator will provide an introduction to the test volunteers.
Thank you for agreeing to participate in the usability evaluation and pilot of the CCSM CDS Dashboard. My name is [evaluator name] and I am assisted by [assistant name]. We are part of a team developing clinical decision support software which implements the latest clinical guidance regarding the screening and management of cervical cancer. This effort includes the development of a ‘dashboard’ which will display the relevant patient history associated with the recommendation following the latest guidelines.
The purpose of this usability testing is to gather feedback and suggestions for improvements.
Your participation is completely voluntary, and you may skip any questions or stop the evaluation at any time. Your performance or knowledge is not being evaluated, only the dashboard itself. We welcome your honest feedback.
The evaluation should take approximately one hour, and you may ask questions at any time.
With your permission, we would like to record this evaluation session. This recording will be used internally only by the development team for purposes of improvement to the dashboard and CDS software. Thank you.
Where do you work and what is your current role?
How long have you been performing this role?
Do you routinely make decisions regarding HPV/Pap screening, cervical exams, cytologic testing, or management of cervical cancer?
What is your prior experience with clinical decision support software?
The usability test will look at the CCSM CDS clinician dashboard. This is a dashboard launched from the electronic health record application and displays a care recommendation based on the latest cervical cancer screening and management guidelines.
Web application
[Test participants will be provided with a selection of sample patients and be instructed to view specific patients for testing purposes.]
After opening the dashboard, Is the information shown clearly labelled and easy to read? Is anything overlapping, obscured or not easily readable?
Is the information shown in the dashboard in two main columns, or is it shown in one long page? Does resizing the window automatically adjust the layout to fit the extra space?
Patient identification information is shown on the dashboard. What information do you look at to confirm that the display shown is for the correct patient?
What portions of the patient's medical history are visible? Do you think the items shown are relevant for cervical cancer screening and management?
Are there any items that you expected to see which are not shown?
Are you able to change the sorting of the items in the patient's medical history? What item(s) are the most recent in the patient's medical history? What item(s) are the oldest?
What would you do to see more information about an item in the patient's medical history?
What would you do to confirm with the patient that the history shown is correct?
If you are aware of a relevant item in the patient history that is missing from the list, what steps would you take to add the missing item?
What is the patient's most recent HPV result?
What is the patient's most recent Pap result?
Are there any items in the patient history which are incomplete or do not have a result shown? How can you tell if an item is incomplete?
Can you find the recommendation suggested by the CDS?
What information is shown in the Recommendation tab?
Is the description in the recommendation tab consistent with the recommendation? Does the text adequately explain the recommendation?
Is a risk table shown below the recommendation tab?
If so, which row is highlighted in the table?
Do you think this is an appropriate risk table to show based on the patient's medical history? Is the table shown consistent with the patient's screening history?
What other tabs are shown adjacent to the Recommendation tab?
What information is shown on the Reference tab? Is there any additional information you would find useful to have on this tab?
The recommendation of the clinical guidelines does not consider all items in the patient's medical history equally - for example, more recent test results are given priority over than older results, and abnormal results are more influential than 'normal' screening results. Can you find which items in the patient's history were used to arrive at the recommendation?
What are your overall impressions of the CCSM CDS dashboard?
What features of the dashboard do you find useful in making a care decision for a patient? What information on the dashboard would you use when discussing care options with the patient?
Are there any missing items that you feel would be useful to have added to the dashboard that are directly involved in making care decisions for a patient with regards to cervical cancer screening and management?
Are there any features of the dashboard which are inconsistent or in conflict with the electronic health record interfaces you are familiar with? (for example, "Lastname, Firstname" vs "Firstname Lastname" or differences in terminology and labeling, such as "Pap Test" vs "Pap Smear")
Thank you for your participation. Your feedback is greatly valued. We are going to continue testing and review the user feedback to help determine what changes and improvements may be beneficial.
Please feel free to contact us if you have any further ideas for improvement or experience any usability issues not covered here.
Public reporting burden of this collection of information is estimated to average 60 minutes per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to CDC/ATSDR Reports Clearance Officer; 1600 Clifton Road NE, MS D-74, Atlanta, Georgia 30333; ATTN: PRA (0920-1050).
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Michael R Nosal |
| File Modified | 0000-00-00 |
| File Created | 2025-05-19 |