A Brown University professor pulled AI out of the final exam of his economics course, and the class average dropped from 96% on the take-home assignments to 48.6% on the proctored final. Eighteen students dropped the course altogether, and nineteen more failed outright.
The Gist
- Same students, same class: 96% average with AI at home, 48.6% in a supervised room without AI.
- The professor rewrote the questions after testing them himself in ChatGPT and getting near-perfect answers.
- Two other studies back the pattern: homework scores rise, exam scores fall when AI is around.
- The gap is not really about grades; it is about whether students still know how to think without a chatbot next to them.
Have ChatGPT Recap This Article
ChatGPTMeet the Brown Professor Who Ran the Experiment
The name is Roberto Serrano, and he teaches economics at Brown University in Providence, Rhode Island. His course is not a niche seminar. It is a standard undergraduate class, the kind hundreds of students take every semester in every university around the world.
Serrano had been noticing something strange for two semesters. Take-home assignments came back nearly perfect, class after class. Averages in the 90s. Sophisticated proofs, cleanly typed. But the same students, during in-class discussions, struggled to explain the very ideas they had supposedly demonstrated a week earlier.
So he ran a test. He rewrote his final exam questions, then pasted every question into ChatGPT to see what came out. The answers he got back were, in his words, nearly identical to the top student submissions. Same phrasing. Same convoluted proofs when a simpler one would work. Same tiny rhetorical tics.
The final exam was then held in a room with no phones, no laptops, no tabs open. Just paper and a pen. The average score collapsed from 96% to 48.6%. Eighteen students dropped the course rather than take the exam. Nineteen more took it and failed.
How the Numbers Line Up With Other Studies
One professor and one class would not be enough. But the Brown result lines up with two larger studies that have been running for over a year, and the shape of the numbers is remarkably consistent.
In China, a research team followed 26,000 university students across many disciplines. Homework performance rose 18% after ChatGPT became widely available on their campuses. In-class exam performance fell 20% over the same period. Same students, same schools, opposite curves.
At UC Berkeley, the pattern showed up in a different form. In courses that lean on written work and coding, the share of students getting an A grade jumped 13 percentage points after ChatGPT arrived. When the same courses added mandatory in-person components, that grade bump largely disappeared.
Read together, the three studies tell a simple story. When students can use AI, their output looks stronger. When they cannot, their unaided performance is usually weaker than what previous cohorts, pre-ChatGPT, were doing at the same stage. The AI is not making them smarter. It is making the visible artifact of their work smarter, while the underlying skill quietly erodes.
Keep learning on AI Noobies:
- Claude Does Office Work Way More Than Code, Anthropic Says
- Claude Fable 5 Wrote 1M Lines of Code in 11 Days
- Muse Spark: Meta’s New AI Model That Uses Your Apps
What Actually Happens When You Learn With a Chatbot Next to You
Nobody in the studies claims students are cheating in the classic sense. Almost everyone is doing what feels natural: they are studying next to a tool that is very good at doing the harder half of the work for them. The problem is that the harder half is where the learning happens.
Try it yourself with a piece of writing. When you draft an email from scratch, you struggle to open, then you find the angle, then you tighten. When you ask a chatbot to draft the same email, you skip those three steps. The output looks the same, but the muscle that would normally build up during those three steps just sits.
Serrano puts it in a specific way. The convoluted proof his top students turned in was not their preferred way of solving the problem. It was ChatGPT’s preferred way. When those same students had to answer without the tool, they did not fall back on a simpler proof of their own, because they never built one. They were left with nothing.
How to See This Pattern in Your Own Daily AI Use
You do not need to be a student to feel this effect. Anyone who has started using ChatGPT for a task they used to do by hand has almost certainly experienced a smaller version of the same drift. There is a simple test you can run this week to check.
Pick a task you used to do well before you had access to AI. Writing a project brief. Summarizing an article. Drafting a difficult email. Then, for one day, do it without opening any AI tool. Notice how long it takes, and how confident the draft feels compared to the AI-assisted version.
If it feels dramatically harder than it used to, that is your signal to rebalance. The Brown class did not fail because ChatGPT is bad. They failed because they let it do the parts of the work that were building their brain. A healthy pattern for beginners looks more like this: do the first draft by hand, then use AI to sharpen it. Not the other way around.
Employers are already noticing the pattern too. Recruiters have started running exercises during interviews that ask candidates to think out loud on paper. What Serrano tested in his classroom is going to be tested, in a different form, on every desk where AI meets skills. The question is not whether AI will be around. It will. The question is whether you kept the muscle to work without it when you have to.
Stay tuned on AI Noobies.
About the Author
