AI for social science methods

During Fall 2025, I am teaching a new course on incorporating generative AI into social science research methods. I will post my materials here, along with notes about what did and did not work. If you are considering teaching this type of course, or already teaching it, I would love to hear from you. My hope is to help build a community of social scientists teaching about AI as a research tool. 

Course materials

I have compiled an initial reading list and created progamming-based activities for students. These are all works-in-progress. If you want to share ideas and/or comments about these materials, please feel free to send me an email.

Much of the course’s content builds on a special issue of Sociological Methods & Research that I co-guest-edited with Thomas Davidson (Rutgers University). The special issue can be found here. The course’s development is supported by Yale’s Poorvu Center for Teaching and Learning and Hridhay Suresh (Yale University) assisted with the initial construction of the activities.

Activities

The files for the programming-based activities are available through the following links. These are the first versions, and I will update them as I teach the course.

  • An activity about prompting: PDF and R Markdown
  • An activity about using LLMs to generate human-like data: PDF and R Markdown
  • An activitiy about measurement and bias: [in progress]

Reading list

My initial reading list is below. It is intended for advanced undergraduates and graduate students in social science majors and programs. As I teach the course, I will record notes about how students respond to each reading. You can find these notes here.

Using AI in the social sciences is a rapidly-growing area of research. There are many excellent papers not included in my current reading list. This is mainly because I am trying out combinations of papers that, first, cover a range of topics and emerging debates (in one semester) and, second, are accessible and interesting to students. If you have suggestions of readings to add, please feel free to email me and I will update the list.

  • Background and overview

    • “Introduction to Neural Transfer Learning With Transformers for Social Science Text Analysis” by Wankmüller in Sociological Methods & Research (2022) link
    • “Start Generating: Harnessing Generative Artificial Intelligence for Sociological Research” by Davidson in Socius (2024) link
    • “Integrating Generative Artificial Intelligence into Social Science Research” by Davidson and Karell in Sociological Methods & Research (2025) link

  • Prompting

    • “From Codebooks to Promptbooks: Extracting Information from Text with Generative Large Language Models” by Stuhler, et al. in Sociological Methods & Research (2025) link
    • “Chain-of-Thought Prompting Elicits Reasoning in Large Language Models” by Wei, et al. on arXiv (2023) link
    • “Concept-Guided Chain-of-Thought Prompting for Pairwise Comparison Scoring of Texts with Large Language Models” by Wu, et al. on arXiv (2025) link
    • “Survey on Prompt Tuning”, by Li, et al. on arXiv (2025) link

  • Prompting applications to text analysis

    • “Updating the ‘Future of Coding’: Comparing Generative AI Models to Other Text Analysis Methods” by Than, et al. in Sociological Methods & Research (2025) link
    • “Large Language Models for Text Classification: From Zero-Shot Learning to Instruction-Tuning”, by Chae and Davidson in Sociological Methods & Research (2025) link

  • Simulation and silicon subjects

    • “Out of One, Many: Using Language Models to Simulate Human Samples” by Argyle, et al. in Political Analysis (2023) link
    • “Balancing Algorithmic Fidelity and Alignment in Silicon Sampling Research Methods” by Lyman, et al. in Sociological Methods & Research (2025) link
    • “Take Caution in Using LLMs as Human Surrogates” by Gao, et al. in PNAS (2025) link
    • “Synthetic Replacements for Human Survey Data? The Perils of Large Language Models” by Bisbee, et al. in Political Analysis (2024) link
    • “Machine Bias: How do Generative Language Models Answer Opinion Polls?” by Boelaert, et al. in Sociological Methods & Research (2025) link
    • “Simulating Subjects: The Promise and Peril of Artificial Intelligence Stand-Ins for Social Agents and Interactions” by Kozlowski and Evans in Sociological Methods & Research (2025) link

  • Measurement and statistical bias

    • “How to Use Generative AIs for Image Analysis in the Social Sciences: Design-Based Supervised Learning” by Maranca, et al. in Sociological Methods & Research (2025) link
    • “The Mixed Subjects Approach: Treating Generative AI as (Potentially) Informative Observations in Experiments” by Broska, et al. in Sociological Methods & Research (2025) link

  • Models’ biases

    • “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?” by Bender, et al. (2021) link
    • “AI Generates Covertly Racist Decisions about People Based on Their Dialect” by Hofmann, et al. in Nature (2024) link
    • “Explicitly Unbiased Large Language Models Still Form Biased Associations” by Bai, et al. in PNAS (2025) link

  • (A few) applications and implications

    • “Durably Reducing Conspiracy Beliefs Through Dialogues with AI” by Costello, et al. in Science (2024) link
    • “Testing Theories of Political Persuasion Using AI” by Argyle, et al. in PNAS (2025) link
    • “Estimating Wage Disparities Using Foundation Models” by Vafa, et al. in PNAS (2025) link
    • “Understanding the Success and Failure of Online Political Debate” by Heide-Jørgensen in Science Advances (2025) link
    • “The Ontological Politics of Synthetic Data: Normalities, Outliers, and Intersectional Hallucinations” by Lee, et al. in Big Data & Society (2025) link
    • “Characterizing the Impact of Participants’ Use of Generative AI on Open-Ended Survey Responses in Social Science Research” by Zhang, et al. in Sociological Methods & Research (2025) link