August 10, 2015
How do firms respond to quality disclosure requirements?
Evidence from performance rankings for U.S. airlines
An aircraft being readied for takeoff. A flight must arrive at the gate less than 15 minutes behind schedule to be considered "on time" for the purposes of the performance rankings published by the U.S. Department of Transportation.
Leonard Zhukovsky/Bigstock
Disclosure requirements can help consumers who might have a hard time figuring out which companies offer the best product in a market. If a customer can’t determine quality until he has already made a purchase decision, a firm has incentive to skimp on quality if it saves money without significantly cutting into the customer base.
A quality disclosure mandate that requires companies to provide information on the quality of their products can be useful, but can also be ineffective or counterproductive if it induces those companies to game the system—unintended effects have been documented in fields ranging from education to health care. An article appearing in the May 2015 issue of theAmerican Economic Journal: Microeconomics studies a quality disclosure requirement in the commercial aviation sector.
In Quality Disclosure Programs and Internal Organizational Practices: Evidence from Airline Flight Delays, authors Silke Forbes, Mara Lederman, and Trevor Tombe analyze millions of detailed flight records provided by major U.S. airlines to detect the impact of a quality disclosure initiative. In 1987, the U.S. Department of Transportation began requiring large airlines to submit flight records, and publishes monthly rankings for airlines based on the percentage of flights that were “on time”, which by their definition includes any flight that makes it to the arrival gate less than 15 minutes after scheduled arrival. These rankings typically receive wide coverage in the media, but reports rarely provide much context about how the data is collected or the definition of timeliness.
If airlines care about public perceptions of their on-time record, they have incentive to reduce delays, but only in a very specific way—if they can incrementally speed up flights that might otherwise have been 15 or 16 minutes late, they can leapfrog other airlines in the rankings without making any sort of deep improvements in their operations or meaningfully reducing overall delays.
Disclosure programs create market-based incentives for firms to improve quality. Yet, quality production is often the result of large and complex sets of decisions made by individuals within a firm.
Forbes et al. (2015)
The authors emphasize the role of organizational behavior in understanding a firm’s response to such a stimulus. The typical large airline covered by this reporting requirement employs thousands of workers scattered across hundreds of U.S. airports, and flights can be delayed for a variety of reasons that are hard to foresee. Even if upper management at an airline decides to make timeliness a priority, airlines might only be able to respond to the incentives slowly, if at all. Then again, airlines have developed “mission control centers” to monitor the status of hundreds of flights simultaneously and make on-the-fly decisions about which flights need to be rerouted or sped up.
To detect manipulation of the on-time threshold, the authors must identify flights that would have been slightly more than 14 minutes late, but for the intervention of airlines. They look at each arriving flight’s wheels-down time at its destination airport and use historical data about that flight and that airport to predict the time it will take to taxi to the arrival gate. For instance, a flight that touches down on the tarmac nine minutes after its scheduled arrival, and which is expected to take six minutes to taxi to the gate, would have an expected delay of 15 minutes.
An analysis shows that flights with an expected delay of exactly 15 minutes at the time when they land are significantly more likely than the average flight to arrive at the gate exactly one minute earlier than expected. These flights systematically take less time than others to taxi to their arrival gate, presumably because airlines are adept at reassigning gates and ground crews to these crucial threshold flights. This is almost certainly attributable to the incentive created by the rankings.
On-time performance rankings induce gaming by airlines
Distribution of flight delays for Continental flights before and after an employee bonus for on-time performance was instituted in 1995. Flight delays bunch at the on-time threshold, but only once the program is in place.
Source: adapted from Figure 3 of Forbes et al. (2015)
But “threshold behavior” was not uniform across firms, despite the outward similarities between this group of ten major U.S. airlines. It is hard for outsiders to observe all of the relevant characteristics of an airline that might make it more or less likely to engage in this practice, but the authors are able to collect information on the existence and timing of several employee bonus programs tied to the on-time rankings. Between 1995 and 2009, five airlines established conditional monthly bonuses for all employees that only kicked in if the airline attained a certain level in the performance rankings. In some cases, but not all, the bonuses seemed to induce threshold behavior where it didn’t exist before (see figure above).
Basic economic reasoning suggests that the monthly bonus would not be a strong inducement for any particular employee to work harder on any specific flight because of the free-rider problem. The chance that any one flight will make the difference in the rankings (which are based on data from about 500,000 flights over the course of each month) is miniscule, so gate crews could sit back and hope that other crews at other airports around the country work hard enough to earn the bonus for everyone. Nevertheless, the incentive seemed to be effective for some airlines, possibly because they were instituted in conjunction with other reforms that the authors did not observe.
Recent years have seen the advent of more reporting requirements including flight-specific timeliness data for online purchases, with data on 30-minute delays. But this could induce airlines to start gaming the 30-minute threshold as well, and even a more comprehensive ranking that incorporated information about a wider range of delays might discourage airlines from servicing some airports that are more prone to delays, like Newark and O’Hare, unless they receive extra “credit” in the rankings for doing so. The authors conclude that understanding how firms respond to these incentives—and why some airlines did not seem to respond at all—will help guide better quality disclosure rules. ♦
"Quality Disclosure Programs and Internal Organizational Practices: Evidence from Airline Flight Delays" appears in the May 2015 issue of the American Economic Journal: Microeconomics.