The Home Health Care CAHPS® Survey
Part B
Collection of Information
Employing Statistical Methods
TABLE OF CONTENTS
Section Page
B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS 3
B.1 Potential Respondent Universe and Sample Selection Method 3
B.2 Information Collection Procedures 6
B.3 Methods to Maximize Response Rate 7
B.5 Statistical Consultation and Independent Review 8
This PRA requests renewal of the national implementation of the Home Health Care CAHPS® (HHCAHPS) Survey with a proposed shortened instrument, conducted by approved HHCAHPS survey vendors doing the survey on behalf of Medicare home health agencies (HHAs). In this statement, we provide details on the statistical methods we used to support making updates to the HHCAHPS Survey instrument.
In 2022, CMS conducted a mode experiment to test the proposed survey instrument. During this most recent mode experiment, we observed small survey mode effects that we did not observe during previous mode experiments. In addition, we also observed differences in results based on patient characteristics, specifically whether the patient lives alone, does the patient self-report fair to poor health, does the patient report poor mental health status, the age of the patient, educational level, whether the survey was completed by a proxy, and the language in which the survey was completed. Each time we update our public reporting file, we will adjust it for the patient characteristics in that file and apply the mode adjustment. We anticipate that while the patient mix adjusters may change with every file, the mode adjustment will remain constant. We will post the patient mix adjusters and the mode adjustments on our website, https://homehealthcahps.org along with downloadable databases of HHCAHPS data. The downloadable databases are also on the Provider Data Catalog on www.cms.gov.
The sampling requirements for the national implementation will remain unchanged and are described below.
For the national implementation, HHAs assemble a census of their patients (both current and discharged) for the sampling window, defined as a calendar month. Every month, each HHA submits a file to its contracted survey vendor containing patient information for all patients to whom the HHA provided skilled home care during the sampling month. The national survey is fielded on a rolling basis, and the results for each quarter are merged with data from the three immediately preceding quarters and analyzed. The sample frame for the national implementation is assembled at the level of the CMS Certification Number (CCN), and the CCNs (Medicare home health agencies) comprise the units of comparison for HHCAHPS Survey results publicly-reported on the Care Compare tool on the Medicare.gov website every three months, in the months of January, April, July, and October.
Each HHA’s sampling frame contains all the patient data needed for both fielding the survey and analyzing the data for public reporting. The HHA’s survey vendor reviews the frame and excludes any patients who are not eligible to participate in the HHCAHPS Survey. Patients are ineligible for the survey if they…
are receiving hospice or are discharged to hospice,
are deceased when the sample is drawn,
are under 18 years of age at the end of the sample month,
did not have at least one skilled home health visit in the sample month and at least two skilled home health care visits during a two-month look-back period covering the sample month and the prior month,
are maternity patients,
are “no publicity” patients,
are receiving only nonskilled (aide) care,
are state-regulated patients, or
are patients who were sampled for HHCAHPS during the previous 5 months.
HHCAHPS sampling methodology includes a requirement that a patient not be sampled more than twice a year. This requirement is intended to reduce burden on individual patients and to increase the probability of response.
For the national implementation of the HHCAHPS Survey, each participating HHA sends a patient sample frame each month containing information to its contracted survey vendor about each patient who received home health care during the sample month, with sufficient information for the survey vendor to determine exclusions and with information needed for both fielding the survey and for patient-mix adjustment. The survey vendor removes from the sample frame those patients who do not meet survey eligibility requirements and then draws a random sample of the remaining patients.
Survey vendors working under contract with HHAs are instructed to use a reliable program to generate random numbers for sampling. CMS has continually recommended to survey vendors that they use the free program RAT-STATS, available from the Department of Health and Human Services, Office of Inspector General website, or some other validated sample selection program such as SAS® to select the samples. CMS’s recommended sampling procedure is simple random sampling; however disproportionate and proportionate stratified random sampling may be allowed as some HHAs may want to analyze their own data and view survey results for individual branches. HHAs that deviate from simple random sampling using disproportionate sampling are required to request an exception and to obtain approval from the HHCAHPS Survey Coordination Team and CMS. An exception is permitted if the minimum sample is 10 per strata and the information needed to increase weights are reported to RTI.
Although the national implementation sampling is conducted on a monthly basis (with the survey initiated for each monthly sample within 3 weeks after the sample month ends), data are accrued, aggregated, analyzed, and publicly reported on a quarterly basis. Four quarters of data are used for public reporting with the data from the most current quarter replacing data from the oldest of the prior four publicly-reported quarters. The data are posted quarterly on the Care Compare tool in the Home Health Care area on www.medicare.gov. For four calendar quarters, a minimum of 300 completed surveys is the target for each participating HHA. If an HHA’s patient population is too small to yield 300 completed surveys, the quarterly targets are adjusted so a census is surveyed. The 300 completed surveys needed for analysis is derived from the formula for the precision of a proportion with the estimate at 0.50, the confidence interval of about +/- 0.05, and a confidence level of 95%.1
In the national implementation of the HHCAHPS Survey, the number of patients needed for selection each month to yield a minimum of 300 completed surveys per four quarters (25 per month) is determined by each HHA and survey vendor. The mode of administration of the survey can be an important determining factor, as response rates vary by survey mode. Based on the target number of completed interviews and the national level response rates, estimated sample sizes by mode for HHAs participating in the national implementation of the HHCAHPS Survey are the following:
Survey Mode |
Response Rate |
Sample Size for 25 Responses/Month |
24.0% |
104 |
|
Telephone |
20.7% |
121 |
Mixed |
29.4% |
85 |
Each HHA’s survey vendor uses its experience on other surveys with home health patients and/or other similar populations, the data collection mode, and expected response rates as guides for calculating the monthly sample sizes that they will use for the HHCAHPS Survey. About 79% of HHAs in HHCAHPS use the Mail Mode, 11% use Telephone Mode, and 9% use Mixed Mode (mail mode with telephone follow-up.)
The sampling rate to achieve these sample sizes indicates that HHAs with monthly frame sizes of 85 or below may need to start with a sample equal to the sample frame depending on their chosen mode. That is, all patients who meet the eligibility criteria will be included in the survey sample. For HHAs with larger sampling frames the sampling rate can be reduced, although it clearly will be higher than 50% for all survey modes until the frame exceeds almost 170 eligible patients per month. Monthly sample size rates should not be based on the number of patients who meet survey eligibility criteria in the frames on the first test month, since that month cannot have any patients who are ineligible for the survey because they were sampled in a previous month.
CMS allows three modes of survey administration during the national implementation of the HHCAHPS Survey to give HHAs options in how they would like to administer the survey based on their goals and resources. These three modes are:
Mail-only mode
Mailing a questionnaire and cover letter to all sampled patients.
Mailing a second questionnaire with a cover letter to sample patients who do not respond to the first mailing within 3 weeks after the first questionnaire package is mailed.
Telephone-only mode
Making a maximum of five telephone contact attempts per patient to complete the survey.
Mixed-mode
Mailing of the questionnaire and cover letter to all sample patients.
Conducting telephone follow-up with all sample patients who do not respond to the questionnaire mailing. A maximum of five telephone contact attempts per patient will be made to complete the survey.
An HHA’s vendor must initiate data collection for each sampled patient no later than 3 weeks (21 days) after the close of the sample month. We do allow HHAs to apply for a late fielding request up to the 14th day in the following month. For example, if an HHA was late sending its April patient list to the vendor (between May 21st-May 26th), then the HHA and the HHA’s vendor can apply for a late fielding request, which CMS usually accepts up to June 14th (in this example). The fielding period is 42 days and there are not extensions on that time frame. Again, once data collection begins, it must be closed out within 42 days or 6 weeks.
Survey vendors who wish to become approved to conduct the HHCAHPS Survey on behalf of HHAs complete the HHCAHPS survey vendor training, which provides detailed guidance on the protocols and guidelines for all aspects of survey implementation, from sample selection to data collection and data submission. We post the list of approved HHCAHPS survey vendors on our website, https://homehealthcahps.org. The HHCAHPS Survey Coordination Team additionally requires every vendor to submit a Quality Assurance Plan once they submit survey data to the HHCAHPS Survey Warehouse, which is updated annually. The HHCAHPS Survey Coordination Team conducts site visits to survey vendors annually or biannually. The HHCAHPS Survey Coordination Team is available for technical assistance by telephone or by email for the HHCAHPS survey vendors.
Every effort is made to maximize patient response rates, while retaining the voluntary nature of the HHCAHPS Survey. Each questionnaire mailing includes a cover letter explaining what the survey is about, who is conducting it and why, and the name and toll-free telephone number of a survey staff member that sampled patients can contact if they have questions or need additional information about the survey. For the mail-only mode of administration, our approved HHCAHPS survey vendors must use best practices in survey materials to enhance response rates. These best practices include using a simple font no smaller than 10-point size in the survey cover letters, allowing ample white space between questions in the questionnaire, avoiding a format that displays the questions as a matrix, using a unique subject identification number on the questionnaire rather than printing the sample member’s name, and displaying the OMB number and expiration date on the questionnaire. The second mailing for the mail-only implementation is expected to increase the response rate, as is the telephone follow-up portion of the mixed-mode implementation.
These analyses are done with each quarterly data submission from survey vendors:
Conduct analyses of individual survey items to assess missing data and item distributions.
Perform Ordinary Least Squares regression on each CAHPS measure using national level HHCAHPS data and a set of patient-mix adjusters previously identified from the HHCAHPS Mode Experiment.
Construct the patient mix adjusters to be applied to each home health agency’s HHCAHPS results.
Patient mix adjust the seven publicly reported CAHPS measures for each home health agency.
Assess the effects of patient mix adjustment in the current quarter by comparing to prior quarters to see if results are similar to prior results.
Add (with this proposed survey) a mode adjustment each quarter.
We found a survey mode effect for HHCAHPS in the 2022 Survey Mode Experiment; therefore, we will adjust for survey mode. When mode adjustments are made, it is necessary to choose one mode as a reference mode. One can then interpret all adjusted responses from all modes as if they had been surveyed in the reference mode. CMS will use mail-only as the reference mode for the HHCAHPS Survey because it is the most used mode for HHCAHPS as noted in section B.1 above. In addition to mode adjustments, the following patient mix adjusters were also identified as significant during the Mode Experiment and will be analyzed and confirmed with each quarter of submitted data:
Self-reported overall health status,
Self-reported mental/emotional status,
Education,
Age,
Whether the patient lives alone,
Whether the survey was completed by a proxy, and
Language in which the survey was completed.
This sampling and statistical plan was prepared by RTI International and reviewed by CMS. RTI continues to evaluate the data in many different analyses. Recently, RTI has evaluated the outliers in the HHCAHPS survey data results, and they have analyzed how the questions contribute to our third composite that concerns specific health issues. RTI has additionally evaluated HHAs that receive footnotes in our publicly-reported data. RTI’s primary statistical point-of-contact is Mr. Harper Gordek, MPH, RTI International, Research Triangle Park, NC (919) 541-1231.
1 Many agencies, with a substantial sampling fraction, can achieve a higher precision because of the finite population correction factor.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | The National Implementation of the Home Health Care CAHPS Survey |
| Subject | OMB Supporting Statement for Home Health Care CAHPS Survey |
| Author | Centers for Medicare & Medicaid Services |
| File Modified | 0000-00-00 |
| File Created | 2025-07-04 |