Mar 21 -- The Census Bureau invites comments to OMB by April 28, 2022 regarding the proposed American Community Survey (ACS) 2022 Content Test.
Content testing is conducted by the Census Bureau periodically to improve data quality. The 2022 ACS Content Test will be a field test of new and revised content. Data from the test will help determine if the proposed wording produces data quality that is as good as or better than the existing questions. For new questions, the test will help determine if there are any data quality issues from the proposed question, or if two versions are being tested which one performs better. The results of this test will help determine which new or revised questions will be implemented in the ACS.
The Census Bureau, in coordination with the Office of Management and Budget Interagency Committee for the ACS, solicited proposals for question changes or additions from over twenty Federal agencies. The following topics will be included in the field test: Household roster, educational attainment, health insurance coverage, disability, income, Supplemental Nutrition Assistance Program (SNAP) and labor force questions. Additionally, three new questions will be tested on solar panels, electric vehicles, and sewage disposal. A summary of changes for each topic are as follows:
Household Roster—The roster instructions have not changed since the 1990s while household living arrangements have increased in complexity. The revisions attempt to capture more complex living situations and improve within household coverage, especially among young children and tenuously attached residents.
Educational Attainment—A relatively high percentage of respondents are selecting the response category, “No schooling completed.” Ongoing research suggests that this includes adults who have completed some level of schooling. The revision attempts to reduce the erroneous reports in this category through formatting and wording changes to clarify the response options.
Health Insurance Coverage—The purpose of testing the revised health insurance question is to enhance question reliability and validity. Since implementation in 2008, research has found that Medicaid and other means-tested programs are underreported in the ACS and that direct-purchase coverage is overreported, in part due to misreporting of non-comprehensive health plans and reporting multiple coverage types for the same plan (Mach & O'Hara, 2011; Lynch et al., 2011; Boudreaux et al., 2014; O'Hara, 2010; Boudreaux et al., 2011; Boudreaux et al., 2013). Moreover, revisions to the health insurance coverage question would help capture changes to the health insurance landscape that occurred with and since the passage of the Patient Protection and Affordable Care Act.
Changes to the health insurance coverage question include the following: Reordering some response options and rewording response options for direct purchase, Medicaid, employer, and veteran's health care. A second version of the question will test these same changes along with a change to the format of the question that adds an explicit response category for those who are uninsured.
Disability—The series of six disability questions are being revised to capture population information on functioning in a manner that reflects advances in the measurement of disability and is conceptually consistent the World Health Organization's International Classification of Functioning, Disability, and Health (ICF) disability framework (World Health Organization, 2001). Changes include using graded response categories to reflect the continuum of functional abilities (the current questions use a dichotomous yes/no response), reordering the questions, and modifying question text. Additionally, a new question being tested attempts to capture difficulties related to psychosocial and cognitive disability in addition to problems with speech.
Income and SNAP—The Census Bureau is conducting research to determine the feasibility of using administrative sources as a replacement or supplement to the income questions currently fielded on the survey. Administrative data sources on employment, income, and public assistance benefits from the Internal Revenue Service, Social Security Administration, and state administrative offices could meet the agency needs for many types of income, transfer benefits, and employment data. If administrative data use in ACS production is implemented, it could provide far-reaching benefits for multiple ACS topics including income, SNAP, and employment.
To better align with administrative data sources on many types of income and transfer benefits, we are testing a change in the reference period from the “past 12 months” to asking about the prior calendar year. Aside from the reference period change, overall instructions are being updated along with instructions for public assistance and retirement income, question wording is being changed for self-employment, public assistance, and total income, and rental income is being collected separately from the interest question. Two versions of the income questions will be tested: One with a change in reference period and the question modifications and the other with only question modifications. The SNAP question will only test changes to the reference period, the question itself is not changing.
Labor Force Questions—The changes proposed to the Labor Force question series are linked to the changes proposed to the Income series of questions, which change the reference period from “during the past 12 months” to asking about the prior calendar year. Changing the reference period to the prior calendar year will allow the ACS to better align with administrative records, which, if used, could improve the quality of ACS estimates. In order to implement the change in reference period, an additional question is also added for respondents who have worked in the past five years. Aside from the change of reference period, changes to the question instructions will also be tested (in two formats).
Electric Vehicles—This new question asks if there are plug-in electric vehicles kept at the housing unit. By adding this question, we will be able to project future energy sources, infrastructure, and consumer needs for the growing popularity of electric vehicles. The ACS would be the only data source at the housing unit level to adequately make these projections.
Solar Panels—This new question asks if the housing unit uses solar panels that generate electricity. By adding this question, we will be able to obtain data for operational solar panels on a housing unit level across the country. This information will help the Energy Information Administration (EIA) match energy consumption to energy production across the United States.
Sewage Disposal—This new question asks if the housing unit is connected to a public sewer, septic tank, or other type of sewage system. By adding this question, we will be able to obtain consistent data on the decentralized wastewater infrastructure status in rural and other communities. This is needed to protect public health, water quality, and to understand and meet the country's growing infrastructure needs. The ACS is the only available survey that can provide these levels of data in a timely, consistent, and standardized manner.
To evaluate the proposed changes, the 2022 Content Test will include a control and two experimental treatments. The Control Treatment will include the current ACS (production) version of the questions. New topics will also be included in this treatment. The Test Treatment will include the test question version for all topics except household roster. Because changes to within-household coverage can impact results of person-level questions, the household roster question(s) will be the current production version in order to avoid confounding. Changes to the roster question(s) will be tested in a separate treatment, called the Roster Treatment. Additionally, health insurance coverage, labor force, and income will include a second set of experimental questions to be tested in the Roster Treatment.
Each topic will be evaluated using a variety of metrics, including item missing data rates, response distributions, comparisons to benchmarks and administrative data, response reliability, and other topic-specific metrics. Comparisons will be made between the Control Treatment and the experimental treatments. In some cases, the two experimental treatments will also be compared. Results of the test will inform decisions about changing content on the ACS.
ACS 2022 Content Test proposal to OMB:
https://www.reginfo.gov/public/do/PRAViewICR?ref_nbr=202203-0607-008 Click IC List for forms, View Supporting Statement for technical documentation. Submit comments through this site.
FR notice:
https://www.federalregister.gov/d/2022-05937
For AEA members wishing to submit comments to OMB, the AEA Committee on Economic Statistics offers "A Primer on How to Respond to Calls for Comment on Federal Data Collections" at
https://www.aeaweb.org/content/file?id=5806