The Past, Present and Future of the Experimental Method in Economics
Paper Session
Tuesday, Jan. 5, 2021 3:45 PM - 5:45 PM (EST)
- Chair: Hannes Rusch, Maastricht University
Geographical Concentration and Editorial Favoritism within the Field of Laboratory Experimental Economics
Abstract
We examine geographical concentration, scientific quality, and editorial favoritism in the field of experimental economics. We use a novel data set containing all original research papers (N = 596) that exclusively used laboratory experiments for data generation and were published in the American Economic Review, Experimental Economics, or the Journal of the European Economic Association between 1998 and 2018. The development of geographical concentration is examined using data on authors' affiliations at the time of the respective publication. Results show that research output produced by US-affiliated economists increased slower than overall research output, leading to a decrease in geographical concentration. Several proxies for scientific quality indicate that experiments conducted in Europe are of higher quality than experiments conducted in North America: European experiments rely on a larger total number of participants as well as participants per treatment, and receive more citations compared to experiments conducted in North America. Examining laboratory experiments published in the AER more closely, we find that papers authored by economists with US-affiliations receive significantly fewer citations in the first 5 and 10 years after publication compared to papers by authors from the rest of the world.Research Registries: Facts, Myths and How They Can Be Improved
Abstract
The past few decades have experienced an experimental revolution in economics, whereby scholars are now much more likely to generate their own data. While there are virtues associated with this movement, there are concomitant difficulties. Several scientific disciplines, including economics, have launched research registries in an effort to attenuate key inferential issues. This study assesses registries both empirically and theoretically, with a special focus on the AEA Registry. We find that over 90% of randomized control trials (RCTs) in economics do not register, only 50% of the RCTs that register do so before the intervention begins, and the majority of these prereg- istrations are insufficiently detailed to be informative. Our empirical analysis further shows that using other science registries as aspirational examples is misguided, as their perceived success in tackling the main issues is a myth. In light of these facts, we advance a simple economic model to explore potential improvements. A key insight from the model is that removal of the (current) option to register completed RCTs could increase the fraction of studies that register. Furthermore, linking registration to IRB applications can both lower the cost of registration and increase their informativeness.What Is Deception in Experimental Economics? A Survey
Abstract
It is almost a religion in experimental economics that deception is bad. But precisely what constitutes deception is unclear. This issue is a thorny one and is a major methodological concern for experiments both in the lab and in the field. While there is a consensus view that deliberate and explicit lies are not permitted, there are quite a few “gray areas” with respect to practices that omit information or are misleading without an explicit lie being told. In this paper, we report the results of a large (788 respondents) survey of experimental economists concerning various specific gray areas. First, perhaps surprisingly, we find that there is a great degree of heterogeneity in the responses. Second, there is considerable difference in opinions across our seven specific scenarios; in particular, the data indicate a perception that costs and benefits matter, so that such practices might in fact be appropriate when the topic is important and there is no other way to gather data. We also survey former undergraduate students (126 respondents) who had participated in experiments, again finding considerable heterogeneity in views. Compared to researchers, students have different attitudes about deceptive methods in the specific scenarios and are apparently mostly only bothered by such practices when this affects their pay. A real surprise is that few students express awareness of the no-deception policy at their former schools.Does Selection Bias Cause Us to Overestimate Gender Differences in Competitiveness?
Abstract
Experimental evidence suggests there is a substantial difference in the willingness of men and women to compete that could help explain the gender gap in labor market outcomes. The use of volunteer samples, however, raises a question about whether self-selection into experiments biases the estimated difference in competitiveness. To address it, we first measure the willingness of 1,145 individuals to compete in a classroom experiment. We then identify among them the subset of ‘lab volunteers’ by observing who accepts an invitation to participate in lab experiments. To test for the existence of selection bias, we compare the gender gap among lab volunteers to that in the population from which they were recruited. We find that selection causes us to overestimate the gender gap in competitiveness by 16 percentage points in absolute terms and, in relative terms, by a factor of 2 to 3 depending on the econometric model. We also show that selection causes us to significantly overestimate the gender gap in risk attitudes and the tendency of low-performing men to select into competition. We present evidence men and women select differently into the lab, and discuss the implications of our findings for future research.Discussant(s)
Sigrid Suetens
,
Tilburg University
Sherry Xin Li
,
University of Arkansas
Matthias Greiff
,
Clausthal University of Technology
Christina Gravert
,
University of Copenhagen
Lise Vesterlund
,
University of Pittsburgh
JEL Classifications
- C9 - Design of Experiments
- B4 - Economic Methodology