How Much Should We Trust Non-Probabilistic Web-based Surveys on LGBTI People? Evidence from Mexico

Although many web-based non-probabilistic surveys are the only source of data on gender and sexual diversity in many countries, there is little evidence of their accuracy in characterizing LGBTI people. This paper studies how precisely the experiences of LGBTI people are captured by web-based non-probabilistic surveys. We compare several statistics about LGBTI people obtained from a web-based non-probabilistic survey with over 14,000 respondents to statistics derived from a large-scale, nationally representative survey with more than 44,000 respondents that was specifically designed to characterize this population. We find important differences in the relative sizes of the subgroups that are part of the LGBTI population and in their characteristics. Our results illustrate how the self-selection inherent in non-probabilistic web-based surveys can result in a characterization of LGBTI people that assigns them higher educational attainment and better labor market outcomes than the probabilistic benchmark while, at the same time, significantly overstating life dissatisfaction, mental health issues, and the prevalence of self-reported experiences of bullying and harassment. These results suggest that while we can learn something about the direction of the gaps between LGBTI people and the general population on self-reported indicators, researchers and policymakers need to be careful in interpreting the levels of such gaps.