Sophiebeta

Research · Product

Why Sophie is not a chatbot.

A chatbot is built to give answers. Learning happens when the student finds them. So we built a tutor instead.

Published

A sunlit study desk with an open notebook, a mug and a small plant.

A chat box is a great tool for getting work done. For learning, it is the wrong shape.

When we started Sophie, the obvious move was a chat window: the student types a question, the AI answers. We chose not to build that. Here is why, in four findings.

1. A chatbot is built to answer

Chatbots are trained to be helpful: you ask, they reply with a full, polished answer. That is exactly what you want at work. In school, the answer is not the point. The effort of finding it is.

Psychologists have known this since 1978: people remember what they produce themselves far better than what they only read. A tool that produces everything for the student skips the part that teaches.

2. Students use it as a shortcut

Give a student a chat box, and most of the time they ask for the answer.

Anthropic studied about 575,000 student conversations with its own chatbot. Nearly half (about 47%) were students "seeking answers or content with minimal engagement." These were university students, and the company that makes the chatbot is the one saying it.

3. The shortcut costs grades

The clearest test so far comes from Wharton. Hamsa Bastani and colleagues gave nearly 1,000 high school math students access to GPT-4 during practice.

  • During practice, students with plain ChatGPT solved 48% more problems. It looked like a win.
  • On the exam, with no AI, the same students scored 17% lower than students who never had it.

Their messages showed why: most students simply asked for the solution. The researchers also tested a careful version, told to give hints instead of answers. It stopped the damage. It did not beat the students who practiced alone. A polite chatbot is still a chatbot.

4. What works is a tutor, step by step

In 2011, Kurt VanLehn compared decades of tutoring studies. Systems that only checked final answers helped a little (an effect size of 0.31). Systems that worked with students one step at a time helped more than twice as much (0.76), almost as much as a human tutor (0.79).

AI can get there too, when it is built as a tutor. In 2025, a Harvard team built a physics tutor that guided students through each step and held back full solutions. Students learned about twice as much as in an active-learning class, in less time.

Same technology, opposite results. What changes the outcome is the design around it.

So we built a learning product, not a chat window

Sophie is a whole tutoring session, built around the student doing the work. Each part answers one of the findings above.

  • She leads. A chatbot waits for a question. Sophie asks how the student is doing, and together they make a short plan for the session.
  • The student works on paper. A phone on a stand shows Sophie the page. She follows every line, not just the final answer, so she can tell a misunderstanding from a slip.
  • Hints, never the answer first. When a student is stuck, Sophie makes the step smaller and gives a small hint, then lets them try again.
  • They talk. Sophie speaks and listens. Students explain where they got stuck in their own words, which is part of the thinking.
  • The screen teaches. A graph, a worked step, a short lesson or a reference sheet appears when it helps, then gets out of the way.
  • She remembers. Sophie keeps notes on how each student learns, and writes the parent a short letter after every session.

A chatbot makes homework faster. A tutor makes the student better. We built the tutor.

Sources