StudentBench Study Finds AI Tutors Match Humans on GRE Gains, at Far Lower Cost
The paper introduces StudentBench, a large-scale platform (175,000+ student-AI messages) comparing AI tutoring, human tutoring, and no tutoring on GRE Quant and Verbal learning gains across 2,383 participants. The authors report statistical equivalence between AI and human tutoring (p=.015), with the best AI tutors outperforming humans in five of seven domains and one AI tutor matching human gains at roughly 918x lower cost; a second study has human experts rate AI-generated lesson plans and practice problems. On Twitter, commentary centered on methodological skepticism, with one widely shared thread flagging that the study excluded students who didn't sufficiently engage with the AI tutor, questioning whether this filtering inflates the apparent parity between AI and human tutoring.
Discussion: 1 tweets from 1 authors · @ddmeyer