The National Science Foundation's National Center for Science and Engineering Statistics (NCSES) released two studies utilizing the new definition of innovation in Oslo Manual 2018, which replaces that in Oslo Manual 2005.
1) Innovation Data from the 2019 Annual Business Survey -- Excerpts:
Nearly a third (30%) of the estimated 4.8 million for-profit companies with at least one employee introduced an innovation during 2016–18 (table 1). Business innovation is defined as a new or improved product or business process that differs significantly from a firm’s previous products or business processes. To be an innovation, the product must have been introduced to the market or the business process must have been implemented by the firm. This definition of innovation is based on guidance in Oslo Manual 2018, a joint publication of the Organisation for Economic Co-operation and Development (OECD) (of which the United States is a member) and the Statistical Office of the European Communities (Eurostat) and provides a common framework for measuring innovation. The Annual Business Survey (ABS) provides a comprehensive view of the incidences of innovation in the United States.
Research and survey revisions have made the concept of innovation measurement much clearer for respondents to report in a meaningful way. The Oslo Manual sets forth a framework to develop a statistical approach to support the measurement of innovation in firms. Changes were made in Oslo Manual 2018 to both the definition of innovation and the types of innovation. These changes were informed by work conducted by OECD and Eurostat and the National Center for Science and Engineering Statistics (NCSES) in collaboration with methodologists at the Census Bureau. Effectively, the Oslo Manual 2018 defines business innovation as the following: a new or improved product (goods or services) or business process (or combination thereof) that differed significantly from the business’s previous products or processes and that has been introduced on the market or brought into use
This was changed from the description of innovation set forth in the 2005 Oslo Manual: the implementation of a new or significantly improved product (good or service), or process, a new marketing method, or a new organizational method
Beginning in 2012, OECD started a review of the 2005 Oslo Manual definition and concept of innovation by conducting cognitive interviews of business respondents. These interviews were conducted in multiple countries, and in the United States the work was led by NCSES. The cognitive interviews focused on the definition of innovation and types of innovation, as well as novelty and innovation activities.
Overall, respondents across all countries appeared to agree with the Oslo Manual approach to require that an innovation is implemented in the marketplace but that an innovation is not required to be successful. However, U.S.-based companies appeared to be more likely to restrict their interpretation of innovation to cases in which it results in some form of technical, commercial, or financial success. That is, in their responses in cognitive interviews, U.S. respondents focused more on outcomes that can result from innovation, including lower costs and higher efficiency, than did respondents in other countries. All respondents, regardless of location, focused on new and improved products or processes or on use of technology; most U.S. respondents indicated technology is not integral to innovation. The NCSES cognitive interviews showed that the term “significantly improved” was a contentious aspect of the 2005 Oslo Manual definition, with several respondents considering the term to be too ambiguous and lacking precise criteria for identifying innovation. There also was a general sense that something “new only to the company” (rather than “new to the market”) is not sufficient to warrant being termed an innovation but rather an imitation. (These findings resulted in specific guidance in U.S. innovation surveys that products or processes need only be new or improved for the business and that innovations can fail or take time to prove themselves.)
The earlier (2005) definition of innovation made a distinction between four different types of innovation: product, process, marketing, and organizational. The cognitive research described above found that although some aspects of innovation were easy to understand, such as product innovation, others were more problematic. Specifically, the distinction between process and organizational innovation was difficult for respondents.
When respondents were asked for examples of innovation, they often provided examples of product innovation and very seldom provided examples of marketing or organizational innovation. When asked for examples of organizational innovation, respondents often gave examples that were process innovation. The confusion between process, marketing, and organizational types of innovation led to the development of the composite “business process innovation” term used in the Oslo Manual 2018 definition. (The concept of composite business process innovations was first implemented on the ABS 2019.)
Impact of Definition Change on ABS Results: Data from the 2019 ABS provide a comprehensive view of the incidence of innovation by businesses located in the United States utilizing the new definitions recommended in Oslo Manual 2018. These survey data represent an estimated 4.8 million for-profit companies publicly or privately held, with one or more employees, and active in the United States in 2018 (see “Survey Information and Data Availability” section).
Thirty percent of the estimated 4.8 million for-profit companies with at least one employee reported having an innovation during the period of 2016–18 (table 1). By contrast, per ABS 2017 more than two-fifths (43%) of the estimated 4.6 million for-profit companies with at least one employee reported having introduced an innovation during the previous period of 2015–17. The difference between the two years can most likely be attributed, to some extent, to the change in the definition but also sampling variation and changes in the questionnaire. For example, in the 2017 ABS, there were four questions asking about each of the four different kinds of innovation, giving respondents in the 2017 ABS more opportunities to say “yes” to innovation. However, in the 2019 ABS, there were only two questions about the different kinds of innovation: product innovation and business process innovation. After accounting for differences in definition, sampling, and survey changes, the two surveys show broadly similar results.
Despite the changes between the 2017 ABS and the 2019 ABS in how process innovation is categorized, as well as the introduction of business process innovation in the 2019 ABS, it is nonetheless possible to compare innovation rates for similar (if not perfectly matched) types of innovation for the two periods. Oslo Manual 2018 provided a crosswalk between the types of innovation covered in the 2005 Oslo Manual with the component activities recommended in Oslo Manual 2018 (table 2). The ABS 2019 collected innovation incidence rates not only for the business process innovation composite but also for the individual innovation types that comprised business process innovation. For the 2017 ABS data collection, process innovation included production, delivery and logistics, and several ancillary services, including purchasing, accounting, and information and communication technologies services.
The process innovation incidence rate was 16% (see table 23 in the 2017 ABS). For the 2019 ABS data collection, process innovation—as contrasted with business process innovation—included production, distribution and logistics, and information and communication systems. With the new definition of innovation and the new questionnaire, process innovation incidence was 14% (table 2).
However, when it comes to the other types of innovation as defined in the 2005 Oslo Manual and cross-walked to the Oslo Manual 2018 definition, the results are noticeably different between the years. In data from 2015 to 2017, the incidence rate for organization innovation was 26%; in 2016–18, it was 9%. For marketing innovation, the rate was 23% in 2015–17, compared to 10% in 2016–18 (table 2).
The data changes in marketing and organizational innovation between the two years are most likely attributed to the changes in the questions that were asked, as well as the change in the definition of innovation. Table 3 presents the questions asked in both the 2017 ABS and 2019 ABS questionnaires.
InfoBrief NSF 22-325:
https://ncses.nsf.gov/pubs/nsf22325
2019 ABS data tables NSF 22-315:
https://ncses.nsf.gov/pubs/nsf22315
2) Using Mechanical Turk in Study of Individual Innovation
Abstract: Innovation is typically studied within businesses, but other sectors of the economy, such as governments and the household sector, can also undertake innovation activity. In order to fully understand innovation at the economy and society level, innovation must be understood beyond the business sector. However, there are numerous challenges to understanding non-business innovation. Individual innovation in particular is assumed to be relatively rare in the general population and thus expensive to measure using probability-based samples. At the same time, relatively few studies have examined individual innovation using non-probability samples. The lack of research on individual innovation leads to uncertainty about the best ways of asking questions related to individual innovation or which topics to prioritize if a probability-based sample is ever used to estimate individual innovation rates. In 2019, NCSES conducted a study of innovation at the individual level using Amazon Mechanical Turk (MTurk) for recruiting respondents and administrating the survey.
NCSES 22-201
https://ncses.nsf.gov/pubs/ncses22201