March 17 -- The National Center for Education Statistics (NCES) invites public comments by April 16, 2021 on its proposal to OMB for the Program for the International Assessment of Adult Competencies (PIAAC) Cycle II 2022 Operational Field Test.
PIAAC is a cyclical, large-scale study of adult skills and life experiences focusing on education and employment. PIAAC is an international study designed to assess adults in different countries over a broad range of abilities, from simple reading to complex problem-solving skills, and to collect information on individuals' skill use and background. PIAAC is coordinated by the Organization for Economic Cooperation and Development (OECD) and developed by participating countries with the support of the OECD. In the United States, the National Center for Education Statistics (NCES), within the U.S. Department of Education (ED) conducts PIAAC. The U.S. participated in the PIAAC Main Study data collection in 2012 and conducted national supplement data collections in 2014 and 2017. All three of these collections are part of PIAAC Cycle I.
A new PIAAC cycle is to be conducted every 10 years, and PIAAC Cycle II Main Study data collection will be conducted from August 2022 through March 2023. In preparation for the main study collection, PIAAC Cycle II begins with a Field Test in Spring of 2021 in which 33 countries will participate with the goal of evaluating newly developed assessment and questionnaire items and to test the operations for the PIAAC 2023 Main Study (2022-2023).
PIAAC 2022 defines four core competency domains of adult cognitive skills deemed key to facilitating the social and economic participation of adults in advanced economies: (1) Literacy, (2) numeracy, (3) reading and numeracy components, and (4) adaptive problem solving. The U.S. will administer all four domains of the PIAAC 2022 assessment to a nationally representative sample of adults, along with a background questionnaire with questions about their education background, work history, the skills they use on the job and at home, their civic engagement, financial literacy and sense of their health and well-being. The results are used to compare the skills capacities of the workforce-aged adults in participating countries, and to learn more about relationships between educational background, employment, and other outcomes.
The Operational Field Test will include tests of survey implementation as well as the functionalities of the tablet device and ease of use of the device and interface for adults with low levels of familiarity with ICT devices. The field test will use a convenience sample of volunteers in the greater Washington DC metropolitan area for a total of 250 completed cases. Volunteers will be screened on relevant demographic characteristics and quotas will be set to achieve a diverse set of study participants. Screening characteristics include: age, gender, level of education, English language proficiency and computer/tablet familiarity. Quotas will be set in each category, and once filled, volunteers with those characteristics will no longer be eligible to participate. Under this data collection, interviews will be conducted in a location with established COVID mitigation procedures.
The PIAAC direct-assessment evaluates the skills of adults in four domains: literacy, numeracy, reading and numeracy components, and adaptive problem solving. These domains are considered to constitute key information processing skills in that they provide a foundation for the development of other higher-order cognitive skills and are prerequisites for gaining access to and understanding specific domains of knowledge.
The PIAAC is an adaptive computer-based assessment. Under this unique design, the computer-based instrument directs respondents to a set of easier or more difficult items based on their answers to a set of locator items. The locator items are designed to assess the respondent’s ability to complete the domain-based items and to assess his or her basic computer skills (e.g., the respondent’s capacity to use the stylus or their finger to highlight text on the tablet computer). As a result, some respondents may not complete all items in the assessment.
In this adaptive design, the instrument uses an algorithm to select assessment items for each participant. This algorithm uses a set of variables that include: i) the participant’s level of education, ii) the participant’s status as a native or non-native language speaker; and iii) the participant’s performance in the locator tasks and on the computer-based items as they advance through the assessment. The key advantage of such an adaptive design is to provide a more accurate assessment of participants’ abilities, while using a smaller number of items than a traditional test design.
At the completion of the BQ, the participant completes a brief tablet tutorial and then begins the reading and numeracy components. Upon completing the components, the participant completes the locator and, based on the locator score, the respondent is routed to the literacy, numeracy or adaptive problem solving items on the assessment.
PIACC website:
https://nces.ed.gov/surveys/piaac/
PIACC Operational Field Test submission:
https://www.reginfo.gov/public/do/PRAViewICR?ref_nbr=202103-1850-003 Click on IC List for survey instruments, View Supporting Statement for technical documentation
FR notice inviting public comments:
https://www.federalregister.gov/documents/2021/03/17/2021-05452/agency-information-collection-activities-submission-to-the-office-of-management-and-budget-for
Point of contact: Holly Xie, PIAAC National Program Manager, NCES Holly.Xie@ed.gov (202) 245-8481
For AEA members wishing to submit comments to OMB, "A Primer on How to Respond to Calls for Comment on Federal Data Collections" is available at
https://www.aeaweb.org/content/file?id=5806