Will the AEA "climate" survey be answered truthfully?

Question

Presumably that's why there are no questions about have you harrassed but why would we believe claims to be a victim?

Answer 1

What I wonder is how one will interpret the answer to  questions about  whether discrimination is widespread. People strongly  say they believe it is, but with diametric opposite claims for who is being favored and  who disfavored.  We've seen that this fall with regard to whether Harvard discriminates or not. Liberals think that if there is any discrimination it's against blacks and hispanics; conservatives think it's against non-hispanic whites and asians.

Comments

My view is that these answers should be interpreted in the context of a strong prior that there is discrimination against women and underrepresented minorities. Why have this strong prior? Because there is a very large body of non survey-based evidence which suggests that there is discrimination. This evidence comes from economics specifically, from academia in general, and from the world at large. I think the survey is useful, but since we are an evidence-based profession we must be sure to use this as one piece of evidence in a very large pool which already points a finger at discrimination (arising from several different sources - structural barriers, explicit and implicit bias, harassment).

Answer 2

Why would we believe any survey results where respondents have an interest in the topic? Is there evidence that questions about sexual harassment are answered more or less truthfully than any other questions?

Comments

I don't understand this point. Don't respondents to surveys have an interest in any issue they care about and which affects them?

---

Yes, they do. Nonetheless, we learn things from survey data. Why would the amount we learn be any different in the AEA survey?

---

I think we are in agreement, then! (Missed the tone of your first comment )

Answer 3

Which questions do you think would be answered untruthfully?

Comments

We know that people answer even non-strategic  anonymised surveys  falsely. For example, Parry and Crossley (1950) found over reporting of ownership of library cards. Here, departmental policies may be influenced by the survey. So it is not just self-image that is at stake. There  are incentives to misreport all questions including demographics. Of course you might say why bother, one survey  will hardly make a difference but  that could be said of filling in truthfully.

---

My understanding is that we think most false answers on surveys come from self-image issues. People under-report traits that they consider negative (or believe others consider negative) and over-report traits considered positive - e.g. ownership of library cards. I don't think the strategic issue is particularly likely to play a role here. Economists are likely to understand that the marginal effect of answering strategically in a big survey is close to nothing...?

---

You could frame the issue as collective self-image. Also, the argument that people won’t respond strategically because they will have a negligible effect on the overall results rather begs the question of why people bother to fill in the survey at all.

As far as I know, there is no evidence on the issue. Perhaps it could be seen whether relatively objective questions yield different answers by gender. “Has anyone in your department been suspended for harassment”. But there are at least two problems. Participation by gender may differ by department. Second, will gender be reported truthfully? Respondents may, for example, want it to be said, not even women think this is a problem.