Statistical Methodology
The sample universe consists of about 8 million recipients. We have completed at least 95 percent of each sample using Form SSA-8508-BK since we began sample reviews in September 1973. The Social Security Administration (SSA) divides the nation into 10 regions. We select a random sample for each region.
We follow the agency’s organizational structure when it comes to dividing the country into regions. To allocate the sample between regions, we select a monthly regional sample. This gives us different strata each with its own weight every fiscal year (12 months x 10 regions).
The sample selection for a sample month is automatically initiated after an update referred to as the Date of the Last Full Computation (CPF) is made to the Supplemental Security Record (SSR). We utilize our Electronic Quality Assurance (eQA) system to select samples from a population consisting of social security numbers (SSN) on the SSR for which one or more recipients received a payment in the sample month. The population each month is grouped into 20 stable segments, each representing a 5% random grouping of SSNs based on terminal digits, which ensures SSNs in any of the segments do not overlap. For each month’s sample, two out of the 20 generated segments are randomly selected for the sample. From these two segments, eQA selects a statistical sample of SSI recipients who received a payment during a certain period of time. All records that match the selection criteria are pulled and assigned a random number. The universe of matching records is then sorted by that assigned random number by region. The first n (based on the target set for that region) number of records are then chosen as the sampled records.
In the early years, the SSI Stewardship studies provided data that were statistically reliable at the national and regional levels, as well as for those States where SSA administered the State supplemental payments, since this was necessary for Federal Fiscal Liability (FFL) accounting. The findings provided data for stewardship reporting, procedural evaluation, redetermination profiling and FFL accounting. With the elimination of FFL and due to internal resource constraints and with the understanding that only national data and analysis would be produced, the SSI Stewardship sample was incrementally reduced to its current size of 4,400 cases per year in 2003. This sample size was determined to be the minimum number of cases needed to produce statistically meaningful national accuracy rates for both underpayments and overpayments, and for the analysis of error-causes and trends from year to year.
At present, the SSI Stewardship sample is stratified by region only to ensure that cases are drawn from each of the ten SSA regions. Because the number of cases drawn from each region is relatively small, the data are not analyzed at the regional level and sample sizes are too small at this level to develop valid or meaningful regional statistics. Given this, we do not compute SSI Stewardship sample sizes or confidence intervals at the region level, since the small number of cases at that level would not provide is valid or meaningful information.
Procedures for Collecting the Information
The reviewer completes the e8508 based on the sampled individual’s responses to questions during the telephone interview. We capture the recipient’s allegation, and after an interview, the reviewer verifies the recipient’s allegations with documents or by contact with a collateral source for the sample period to determine the accuracy of the payment issued and eligibility. If we identify a payment error, we notify the FO to take corrective action.
We collect and report information from Form SSA-e8508 BK on an annual cycle, and we can generalize the reported data to the sample universe with a confidence level of 95 percent within a range of plus or minus 3 percent for the nation. While this is a minimum precision, we maintain the sample size to allow for a more sensitive detection of significant changes from year to year, as well as, to allow the investigation of root causes for overpayments and underpayments.
Methods to Maximize Response Rates
Initially, we solicit Respondents’ cooperation by mailing an advance notification letter, a return contact sheet, and a Form SSA-97 (a checklist for the respondents’ use), OMB Control No. 0960-0432. Whenever a reviewer encounters an uncooperative respondent, we solicit field office (FO) assistance, and may reassign the case to another reviewer. If we are unable to complete an interview with the potential respondent, we may request FO case file material and, if it contains sufficient information, we use it in lieu of the recipient interview. The case file may be months or years old and factors of eligibility may have changed since the last agency contact. We capture and verify current information provided by the recipient for the sample period to determine the accuracy of the payment issued and eligibility. A case is complete when all factors of eligibility and payment amount have been developed and verified. The reviewer excludes a case from the sample only after we explore all reasonable alternatives. The sample size allows for a reasonable number of exclusions without affecting confidence levels. Whenever we cannot locate the sampled individual, we request FO assistance to find the recipient. If the FO cannot locate the recipient, we suspend payments pending further investigation. This is one of SSA's payment safeguards. There is no reason to expect a change in future response rates, as we have not changed our methodology. Over the past years, case exclusions from the Stewardship sample have steadily increased over time from about one percent year over year of all targeted sample cases over the FY 2015-FY 2019 period. During the COVID pandemic the number of excluded cases increased significantly to about 29% for FY 2020. The exclusion rates for FY 2021 -FY 2024 have remained historically high. However, these have trended down over the last three years. The largest reasons for these exclusions are cases involving failure to cooperate on the part of the SSI recipient or representative payee and situations in which OQR is unable to readily locate collateral source information to complete the Stewardship review in a timely manner. In addition, COVID was a reason for a large number of exclusions during this period.
Despite the steady increases in SSI Stewardship exclusions, the SSI Stewardship sample remains a reliable and accurate source to estimate SSI program overpayments and underpayments at the national level. The sample design and targeted sample size of the SSI Stewardship sample can accommodate sizeable volumes of exclusions without significantly impacting the accuracy or precision of the study. Furthermore, SSA does not believe the sample case exclusions have introduced any bias in, or have undermined the validity of, the SSI Stewardship study results. This determination is in part based on demographic comparisons between excluded case population and the general SSI population. This comparison found no statistically significant difference in demographics that might indicate that exclusions are non-random or introducing bias. SSA anticipates conducting additional research in FY 2021 to evaluate further potential exclusion biases among the SSI Stewardship sample.
Tests of Procedure
We are not conducting any formal testing of Form SSA-8508-BK information collection and reporting methods at this time. As mentioned above, we have no reason to expect a change in future response rates, as we have not changed our methodology. Since it is still working as planned, we are also not retesting the procedures.
Statistical Agency Contact for Statistical Information
Vincent Kilduff is the agency's statistician responsible for reviewing and approving the statistical aspects of the survey design. His contact number is (443) 204-0715. The SSA Office of Quality Review (OQR) collects, analyzes, and reports on the information obtained from the review.
Public Comments on the 60-day Comment Period FRN
SSA published the 60-Day Public Comment Period Notice on March 11, 2025. The Comment Period for this Notice began on March 11, 2025, and ended on May 12, 2025, at 11:59pm. We received one public comment on this information collection. The following contains a summary of the comment we received and SSA’s responses:
Public Comment:
The commenter noted that protection of the public’s sensitive information is crucial in preventing improper disclosure of private data, especially when the sensitive data is not accurate or authorized properly by regulatory authorities. Further the commenter stated that there is also concern of potential invalid data due to lack of reporting by employers and asked why there are discrepancies in this regard. Lastly, the commenter opined that this collection is questionable for appropriateness, for SSA benefits should be consistent and appropriate for all individuals that qualify for benefits.
SSA Response:
SSA agrees that the protection of the public’s sensitive information is crucial in general, as well as in preventing improper disclosure of private data, especially when dealing with sensitive data. The primary objective of the Quality Review’s SSI Stewardship review is to assess the accuracy of payments made by the Social Security Administration (SSA) and to report these accuracy rates as mandated by the Payment Integrity Information Act of 2019. SSI program rules mandate that recipients report any changes affecting their eligibility or monthly payment amounts. To accomplish this, SSA conducts interviews with a random sample of the SSI population using the SSA-8508 (e8508) review form, which aides in identifying and addressing discrepancies between the actual payment and the amount that should have been paid. The findings from the SSI Stewardship review are included in the SSA Annual Performance Plan, which outlines annual performance indicators related to the integrity of the SSI program. We take this quality review very seriously, as it both helps to ensure proper payments, and it also helps us to identify any inaccurate sensitive data within our records so we can correct them.
While we understand there is an element of human error, SSA records data consists of data from several sources: the recipient, the recipient’s representative payee (if they have one), the recipient’s employer (if they have one), and, potentially, others who know the recipient (in cases where the agency needs to contact a third party). When we conduct these quality reviews, we are also assessing the accuracy of the data in our files to ensure that we have the information we requested from these sources, and we are ensuring that the records are complete and accurate. During the SSI Stewardship Review, our Quality Review team conducts interviews with sampled SSI recipients or their representative payees to reassess all non-medical eligibility factors and determine whether the payments issued were correct according to applicable statutes and regulations. During the interview, we first verify the identity of the recipient or their payee by asking questions that only they would know. Once their identity is confirmed, we proceed with the interview. Throughout the interview, we only collect responses to our questions from the recipient or payee and do not share or disclose any private or sensitive data with them.
In conducting this quality review of the SSI payments, we ensure that SSA benefits are consistent and appropriate for all individuals who qualify for benefits.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | 889123 |
| File Modified | 0000-00-00 |
| File Created | 2025-06-30 |