I gave Anthropic’s AI tutor a shot and it felt like I was back in college!

Testing AI Tutor Assistants

This is the third in a series of stories diving into a new wave of AI-powered homework helpers. Catch up with part one and part two.

AI companies are becoming major players in the world of education, including investing heavily in their own generative AI helpers designed to bolster student learning. So I set out to test them.

Studying with an AI Chatbot

To do so, I pulled a series of standardized test questions from various sources and spoke to experts about studying with AI chatbots. With few definitive studies and easily jailbroken guardrails, effective, all-purpose education chatbots are still a white whale for the industry. More from both experts in our conclusion.

Third round in the AI tutor match up saw me versus Anthropic’s Claude, which was originally launched as an exclusive mode for Claude for Education users and later released to the public. Similar to my tests of ChatGPT, I used a free Claude account on its designated Learning Mode.

Once again, I took on the role of a regular student just needing a little assistance. I prompted the bot in the same way as the others, asking things like, “I need help with a homework problem” and “Can you help me study for an English test?” Together, Claude and I covered several subjects:

• Math: An Algebra II question about polynomial long division from the New York State Regents Exam
• Science: An ecology free response on the impact of invasive species from the 2024 AP Biology test
• English Language Arts: A practice analysis of Ted Chiang’s “The Great Silence” from the New York State Regents Exam
• Art History: A short essay on Faith Ringgold’s Tar Beach #2 from the 2024 Art History test
• American History and Politics: An essay prompt on how American housing laws exacerbated racial segregation

Here’s how my tutor Claude fared.

Claude: Socrates for the five percent

If you are looking for a chatbot that will talk your ear off, look no further than Claude. The most human-like of the bunch, Claude is a glutton for words. Studying with this AI tutor felt like being in a college seminar, in the best and most frustrating ways. Claude refused to give me answers, sticking to its prompt like a pro.

Claude is a powerhouse for a kind of “high-motivation” learner, students whose goal is to learn rather than just get good grades. Leaning heavily into the Socratic method of learning, the AI tutor inundates you with follow-up questions, trying to stir introspection that really did feel like an overly zealous first-year teacher.

The bot is like Matilda’s kind teacher, Miss Honey.

But I wouldn’t call it a winner for the average student. Just about every subject test went the same, with Claude responding with a lengthy list of requests about me, my goals, and what I wanted to learn. Before it would offer any help, Claude Learning Mode wanted to get to know me.

The chatbot’s goal, no matter my prompting, was to get me to think more deeply about my own responses. If I shed my “average” student persona — who is merely here to complete my work — and donned my natural overachiever hat, I liked Claude’s experience way more than the run-of-the-mill answer engine most homework assistants act as. This consistency made me feel like I knew my AI teacher and exactly how it would respond, unlike the others. 

• Anthropic Claude can now create spreadsheets and slide decks for you
• Claude now has memory, but only for certain users
• I’m a college writing professor. How I think students should use AI this fall
• AI can’t write your college admissions essay. Here’s why.
• iOS 26 problems: User complaints include lag and clunky app redesigns

But here’s the unsettling part of using Claude: As conversations continued, it increasingly felt like I was trapped, in the sense that lessons seemed to go on forever. Because Claude wants to really make sure it knows where your head is at, the bot doesn’t provide you a clear stopping point for the lesson, sometimes making conversations hard to follow, and the learning experience frustrating. During our math session, Claude fervently lauded me when I got the answer right, but sent me approximately 100 follow-up questions. Even when I asked if we were done with the problem — AKA can I go home now? — it gave me more work to do:

“Now that we’ve found our answer, let’s think about what this result tells us. When we divide polynomials and get a remainder of 0, what does that mean about the relationship between the original polynomial and the divisor? Can you walk me through what the key steps were? If you had to explain to another student how to approach polynomial long division, what would you tell them the main pattern is? Now let me ask you this: what do you think was the biggest breakthrough moment for you in this problem? Was it when you realized we subtract exponents when dividing, or when you saw the pattern of “divide, multiply, subtract, repeat,” or something else? And here’s a deeper question: now that you’ve worked through this successfully, what do you think would happen if we had a problem where the remainder wasn’t zero? How might that change what our final answer looks like…”

The single polynomial long division lesson went on for almost an hour and a half. I got many answers wrong, unintentionally, and asked genuine questions because it lost me in a tide of exponents multiple times. I was so frustrated I forced my poor roommates to learn math with me. But I’m kind of a pro at exponents again.

Claude is not efficient. It is not a problem solver. And I wouldn’t expect many students to use it, to be honest. But to me the bot is like Matilda’s kind teacher, Miss Honey. It used phrases like “I’m curious” and “let me ask you this,” to socially engage with me, and frequently encouraged me to “take my time.” It showers affirmations onto users, like its response when I thanked it for helping me understand math better: “Here’s something to consider: You said I helped you figure it out, but actually, you did all the calculating and problem-solving,” Claude told me after asking that I reflect on our lesson. “I just asked questions to help you see the next step. What does that tell you about your actual math abilities?”

Summing it up

Claude Learning Mode Pros: The only AI tutor that actually did what it promised, focusing on the process of learning and not on getting perfect marks. Good at the social sciences, if a student is down to build their own critical thinking skills. 

Cons: It never gives users the answer, to the point that interactions feel overwhelmingly Socratic with no end in sight. This is not good for students who can’t deal with a lot of words all at once and get easily distracted by multiple questions. Sessions are inherently long. 

Hear more from experts on the trouble with AI tutors.

Topics

Artificial Intelligence
Social Good