I think teaching a child to trust an LLM from a formative age is horrifically irresponsible.
If anything, an app should be made where a child learns to correct an LLM's mistakes and learn that it isn't trustworthy.
Actually, better, don't put an LLM in front of children. At all.
EDIT: If a use case is for children who can't afford good education, then use an LLM to make educational materials for children, review them, and make them available for free. After all, the contents are ripped off from human educators anyway.
It's the natural successor to the first round of ed-tech, which both share the same purpose: create future customers. The students who grew up using Google or Microsoft laptops and office products in school go to them as adults. Making children dependent on LLMs at an early age gives you subscribers to your LLM products when they grow up.
I think a Socratic Method-style AI tutor would work, though. It would be less about the LLM providing direct educational material themselves, but making sure the student is thinking about the right questions to direct themselves in learning the material.
How do you ensure the kid is learning the right material? You want them to stay on topic, but a Socratic Method that starts off on one topic can easily branch into many other topics, and then suddenly your student has spent the whole class learning about something other than what they were supposed to.
All learning is good, yes. But a baseline curriculum is necessary to create a ground floor of knowledge for society.
For a current-day LLM, deviation could be measured as the number of tokens since the agent touched on the prompted topic. A specific <<refocus>> token could even be trained so that a harness or other tooling could direct the LLM back towards the prompted topic if needed.
Or something roughly like that. Generally speaking the goal would be for the LLM to bring the topic back on its own while incorporating the tangent, but a hardcoded, "Let's get back on track" button would probably be needed
Just run a loop with another agent that watch the transcript and trigger tool to refocus on the main agent or re-run agent entirely with follow up prompt not visible to the user so it gradually goes back to the right topic.
-> I think teaching a child to trust an LLM from a formative age is horrifically irresponsible.
I largely agree, although with some nuance. One of the most important skills a young child today is going to need as they grow up is the ability to understand what's AI and what isn't, and how to interact with things that are. We actually work hard to make sure a child knows they aren't talking to something real - that they're talking to a computer program.
If part of the comment was on the accuracy of LLMs; our system of multiple harnessed LLMs running in parallel does deliver pedagogically solid and accurate content and experiences. There's a few comments in this thread about LLM hallucinations - We have curriculum specialists constantly evaluating the interactions that Ello delivers, and it just isn't a major factor right now.
So we need to ensure that we're delivering a high-quality, accurate experience for a child, while at the same time making sure that they understand what's reality.
-> Actually, better, don't put an LLM in front of children. At all.
I've responded above to your reason against using AI in education. But to judge whether it's right to avoid using AI with children entirely, we also need to weigh the positive side. So many children get a substandard education that isn't responsive to them and doesn't set them up for success - whether that's a child stuck in a class of 30 being treated as average, or one of the 270 million kids who don't have access to any education at all. Saying "don't put an LLM in front of children" misses the reality of what education actually looks like for so many of them IMO. If your kid gets everything they need from their school, their home, their private tutor and so on, then great.
So our answer is to use the benefits of AI to build technology that teaches - that understands where a child is and gives them the next learning experience they need to keep growing - while making sure they understand they're dealing with a computer program, and without trying to replace the critical role of the people in their lives.
> We actually work hard to make sure a child knows they aren't talking to something real - that they're talking to a computer program.
The problem is not talking to a computer program. Kids have been using computers in school long before LLMs.
The problem is teaching a child to trust what the non-deterministic system is guessing at them. That will carry forward in their life.
You’re teaching them that an LLM is right, not just what it is.
As an aside, I’m also not sure about the certainty of what “making sure” a 4-year old knows is real or not is.
> If your kid gets everything they need from their school, their home, their private tutor and so on, then great.
That’s a powerful emotional point made to shut down criticism. Except it has nothing to do with putting a kid in front of LLMs.
I expressly mentioned this in my comment. You could create a mountain of educational material, apps, and tools with LLM assistance and verify it. Same result.
-> You could create a mountain of educational material, apps, and tools with LLM assistance and verify it. Same result.
I very strongly disagree with this. Teaching is not the same as a bunch of educational material and apps. The use of AI in education does not replace teachers, but it can get much closer to the responsive, learner centered, high-touch learning experiences that teachers provide, and that are proven to deliver the best outcomes.
-> That’s a powerful emotional point made to shut down criticism.
You made a blanket statement - without any substantive points - that we shouldn't put LLMs in front of kids. I was arguing that there are advantages that we should take into account, not trying to shut down criticism.
-> not sure about the certainty of what “making sure” a 4-year old knows is real or not is.
I agree. We're being very cautious here. My co-founder, who is a clinical child psychologist, has a blog post on exactly this you can find it in the footer of our website.)
We’re going to disagree about the quality of LLM learning experiences.
As an educator I’ve used them a lot for myself and in teaching others. Good for one-off topics if you check facts and a lot of human educators have built good quality content that it rips off.
LLMs are rather terrible at an extended progressive curriculum.
That said, the core thesis was simple: it’s dangerous to teach children to trust LLMs.
Not what they are. That they’re right. Which they aren’t. A lot.
You said you largely agreed with that. So, what’s the approach within this service to mitigate that?
I appreciate you responding but you’re throwing a lot of “but X” when the question is “what about Y”.
That's also grammar that's fit to get you harangued and chided by other 4-9 year olds on the schoolyard. Social outcomes are the primary means by which language is meaningful.
This. I would never allow my kids onto a platform like Ello. My kids are starting to appreciate being shielded from social media and technology in their lives as they watch their friends go down dark paths and stuck in feeling like they have to participate there.
I don't think kids, of any age, should be in front of LLMs unless by complete agreement of their parents and only after their parents are made very aware of how this technology came to be, how it can be used and how it cannot be guaranteed to be safe. Ello leading with that is disingenuous from the start and that should tell parents all they need to know up front.
My kids read hundreds of books a year. They love reading, we read with them every single day as they learned. They learned to read the right way and that doesn't involve a screen.
If I may, the argument of "screen" isn't necessarily the right one, the right way might not be the only right way, I have an almost 2-year old daughter and I've bought an e-ink (colored) tablet to show her family pictures, stories and so-on (very sparingly, maybe about 15min a day), we also generate books with images and short stories about what went during this week with the pictures we have and apply a child style to it, and it has been really great so far. It's not about "paper", it's about the software itself and the hardware not being toxic/addictive. In my side, she has a whole library in her playpen with already about 30 books (multiple languages), I would never put her on Youtube or anything like it, but carefully curating the content and showing it on eink, I see absolutely no problem.
Example of page we generate (average "book" has 4-8 pages like this):
https://ibb.co/Ld9mp10v
She loves it, she is engaged into it, this is from an LLM, I don't see the problem with it, I also don't see how it's a problem to generate LLM-made written stories as well, they can be great, personalized and better in many scenarios than random books. I will definitely build learning materials entirely made by AI as well.
Now regarding this project, I feel it's indeed quite dangerous, but I feel as a complementary learning thing, it could be great, for example post-school, 30min of this daily to consolidate learning might be beneficial, I will do so at quite a young age, but with my own softwares & devices of course, not that.
We aren't even living in the US :p - little harmless hallucination from model.
PS: Those animals are from her collection of plush toys which I've individually put as a Sprite for AI usage, I got about 50 plush toys (1 a week in average), mostly used to benchmark her to see if she can recognize the animals from the books she is reading, and she does, all of them, it's a very useful purchase.
This probably isn’t for kids like yours or mine, who have engaged parents who read to them every day and foster learning in more resource-intensive ways. Like it or not, many kids have nothing like this. I’m sympathetic to the idea that they’re better off with Ello than nothing, which is their realistic alternative.
I came here to say this. Part of learning is social interaction. Learning happens on both sides, from the teacher and the student.
In a world of AI everywhere, off-line social interaction will be the major currency. We're all going to crave people... and each other.... The kids who grow up with social skills will be the most successful.
literacy causes profound brain structure changes and psychological effects, and mass literacy results in profound social transformation. which is why people have be trying to prevent the spread of literacy for thousands of years, to this very day.
an LLM is a next token prediction function. of course it matters how you use it, and if you want to use it for an educational purpose, that has to be intentional, so i agree with you on that.
chatgpt and all the other ai powered scams and attention monetization apps are horrible, because they are just using improved technology to accelerate the exploitation of consumers (both adults and children) of media and app platforms.
the solution here is not to say the technology is bad but to use it to create competitive products that are actually beneficial.
> AI is a massive leap forward in providing safe and educational digital experience.
Can you point to any serious studies that sustain that claim? If anything that sounds like a gamifying learning experience, letting children become expert beginners.
Can you point to any serious studies that show the opposite? No. They don't exist. There cannot be any serious studies until there is a generational cohort that can be evaluated for long term impact.
But on a personal level I would vastly prefer my children use even a crappy AI chat app like Deepseek than watch Youtube. If you have young children and pay attention it would be almost impossible to come to a different conclusion.
He didn't make the opposite claim. You are the one that made the claim, meaning you have the burden of supporting it. If there are no studies then no credible claim can reasonably made.
How about the broad anecdotal consensus that Duolingo, a gamified language app, famously does not do much if anything to prepare people for conversational fluency.
I don't disagree but something less bad is better than something more bad if those are the only two options. I only said Duolingo was better than Youtube. Duolingo is nuts. I get my children's spam from them and it is this crazy emotionally abusive manipulative therapy talk nonsense trying to get them to use the app. I hope they are using AI to write that. LOL. I can't imagine the actual human who would do that for a job.
I don't think it's quite as simple as that. LLMs do make mistakes but generally with slightly obscure difficult stuff. They're not going to struggle at all with things a 4-9 year old is learning.
Also it's not like teachers never make confident mistakes.
> They're not going to struggle at all with things a 4-9 year old is learning.
Ask them how many r's are in strawberry.
> Also it's not like teachers never make confident mistakes.
That's an extraordinary false equivalency that is popular around here. Teachers don't make confident mistakes because the student asked the wrong way. Teachers can be held accountable. Teachers can learn. Teachers can love their students.
> There are 3 letter *"r"*s in "strawberry."
>
> Spelled out: s t r a w b e r r y → the r's are the 3rd, 8th, and 9th letters.
> Teachers don't make confident mistakes because the student asked the wrong way.
I don't see why the mechanism of being wrong matters. If anything it's worse when teachers are reliably wrong independently of how you ask the question.
> Teachers can be held accountable.
lol no. Have you ever tried correcting a teacher?
> Teachers can learn.
Good teachers can learn. The ones that make confident mistakes don't. Anyway no learning is needed for 4-9 year old education.
> Teachers can love their students.
That's just such a weird point to make. It is great that my kids' primary school teachers are super kind and loving. But it's not what they're for, and it's definitely not what AI tutors are for. Nobody is claiming this will replace a hug from a parent.
> Anyway no learning is needed for 4-9 year old education.
This is an extraordinarily naive statement. Learning how to teach well takes a lifelong commitment. A topic being basic for an adult does not mean it is easier to teach to a child. In many ways it makes it harder.
> That's just such a weird point to make. It is great that my kids' primary school teachers are super kind and loving. But it's not what they're for, and it's definitely not what AI tutors are for. Nobody is claiming this will replace a hug from a parent.
This doesn't seem weird at all. Developmentally they need (kind, loving) human interaction. That's obviously not the only purpose of teachers, but if you think 7 hours a day without it wouldn't make a difference... just like a computer can't be held accountable, a computer can't have compassion.
No, it's not. The claim that "teacher's make mistakes too" suggests that AI is equivalent to a human teacher in value. I posit other ways in which a human teacher is potentially more valuable.
Also, teachers who love their students tend to want to educate them well.
> lol no. Have you ever tried correcting a teacher?
Do you mean as a student? If so, that's a different dynamic. Because teachers certainly have accountability processes in many institutions.
> Teachers don't make confident mistakes because the student asked the wrong way. Teachers can be held accountable. Teachers can learn. Teachers can love their students.
All of these can be true, and at the same it can be true that a child tutored by a good quality state-of-the-art LLM with a good teaching-focused harness could have better learning outcomes than a child without it. Even if we agreed that a good human teacher is better than a LLM, human teacher's time and attention is limited.
It's a classic example of the kinds of questions LLMs get wrong. There's plenty of others. Not sure your point here. We can easily find things a 4-9 year old will talk about that an LLM will get wrong, hallucinate, etc.
> a child tutored by a good quality state-of-the-art LLM with a good teaching-focused harness could have better learning outcomes than a child without it.
There's a lot of work being done in 'could'. And it's entirely ignoring the dangers.
I'm not saying "no LLMs in education". I'm a technical educator and I give students LLM prompts and agent skills that I've built to help them learn.
This isn't that. We're talking about giving an LLM to a 4-9 year old and saying "this is your teacher".
> It's a classic example of the kinds of questions LLMs get wrong. There's plenty of others. Not sure your point here. We can easily find things a 4-9 year old will talk about that an LLM will get wrong, hallucinate, etc.
My point is that the hallucination rate of modern LLMs, while not zero, is so low that this is no longer an issue in practice.
> There's a lot of work being done in 'could'. And it's entirely ignoring the dangers.
I agree that there are some risks. But for many children, the alternative is not human teacher, but no teacher at all. This considerably changes the risk-benefit calculus, IMHO.
Teaching children is not just about delivering facts to them. The challenge when teaching children is not "do you know the topics that you are teaching them." They challenge is in the pedagogical methods used to get them to learn.
You’re applying human cognition expectations to an LLM. Just because the topic is simple to a human doesn’t mean it improves the likelihood the LLM will behave better. It’s all inference to the machine.
I’m applying my experience with these models, as well as common sense.
What are you applying? Would you even admit LLMs are useful for this if they’re perfect at topics that kids would be learning? Or are your objections actually ideological rather than pragmatic?
I'm not sure how common sense says that LLMs will hallucinate less on topics for children. That's not how they work.
And no, LLMs are not perfect. They're non-deterministic.
And no, even if they were perfect my statement was that it is irresponsible to teach a child to trust an LLM. Even if it was perfect on these topics, as they move forward in education that trust would later fail them.
That is a problem both ideologically and pragmatically. You have to think long-term.
On a side note: ask an LLM to write a story at a 4-9 year old level. Work with it for awhile doing iterations and it will hallucinate a previous detail or change a character name as context rots. Lots of other examples of "basic" topics that still produce hallucinations.
Man, it must be exhausting to have reading comprehension this bad.
Let’s try again: your claim is that LLMs “make up stuff all the time”, so they can’t be used to educate children.
I’m saying that modern SOTA models do not make up stuff all the time as it pertains to the topics a child of 4-9 would learn. That doesn’t mean they’re perfect or will never make mistakes, but there’s zero reason to think such issues would be widespread. And common sense would tell you that perfection is not required for LLMS to be a net positive in education.
I’m sure you’ll have another vapid rejoinder, but this has been boring for awhile and you seem to be enjoying being obtuse, so I’m moving on.
I use them every day for SWE. I cannot remember any hallucination from the last 6 months. It pretty much stopped being an issue from the beginning of this year, if you use top paid models. Free models or lower-tier paid models can still hallucinate, though.
I can use Claude as much as I want at work, and I get hallucinations every day. I'm not even doing anything all that fancy with it, but I get frequent hallucinatory behavior regardless of the model or how I go about things.
Every single time I have asked Claude or another "top tier" offering a detailed question about something I'm an expert in, there's something wrong in the answer. Usually many things.
This is what it seems to be. I've had to review the LLM output from coworkers who have insisted that the "old" models are terrible but the new ones are very good. They are always discussing the latest models and their tweaks and things.
Their output is just.... not very good. For example, it may not fail tests, but it does so for wrong reasons and will fail under different conditions. Or it checks or guards for things in trivially redundant ways, like doing something like `if (x > 5 && x > 3)` that betray its lack of "knowing" what it's doing. And I can't even get answers from them anymore because they just feed anything I say into the LLM and copy/paste the response.
I'm basically being forced to code with LLMs via review through a person proxy. Or maybe they just have it hooked up to read it directly. It's maddening. It's like having a realistic enough chatbot just shuts people's brains off.
It would be extremely valuable to be able to reliably identify this kind of behavioral tendency in an interview setting. Some kind of reliable test that screens for likelihood to go AI psychotic.
> It's a classic example of the kinds of questions LLMs get wrong. There's plenty of others. Not sure your point here. We can easily find things a 4-9 year old will talk about that an LLM will get wrong, hallucinate, etc.
I'm not participating in the debate, but fyi I have just asked haiku, sonnet, opus and fable that question and they all got it right. Tried with the letter s in portuguese word "sussuarana" (mountain lion) too - same thing.
This assumes that children's learning is solely about correctness and obtaining information. There is a whole universe of other considerations, like interacting with other people, with adults, with fallible authority figures. Having a conversation with a teacher and getting the wrong answer teaches you far more about being a human being and living in a human society, than getting facts back from a screen.
Yes, teachers being incorrect is something that should be improved, but reducing human interaction is not an improvement.
"Mister LLM, what should I think about <controversial political topic>?"
Say what you will about the biases of teachers, but I'll take them any day over the propaganda [1] some foreign megacorporation has inculcated into its AI.
This is wildly dangerous comment and I'm glad to see it downvoted.
I have 2 kids. Both of them went through early grades during COVID. The change in learning and dislike of school was immense during that time-frame. My eldest missed out on the most and it's very clear that their age and time with which they should have been in-class and in-person impacted them.
Not only the screen-time aspect of this ignorance, but the sheer inconsiderate approach to what a model even is. It's not a human, it can't cope or understand a child's current state of being. And if you think it can then stay away from K12 tech, please.
Teachers make mistakes, yes. But can you swap a teacher with another teacher without a student knowing? No. You can with a model and that is dangerous. And not only the company. If Ello wants to they can move students to a Chinese model and nobody would know the difference. Or the OAI model they've chosen can be diluted or changed without Ello even knowing.
Kids deserve better. I truly hope Ello fails in its mission because companies shoving models into K12 are truly going to be the stain we all have to deal with decades from now. I am beyond grateful that my kids learned reading the "old fashioned" way and it speaks volumes that both of them read constantly. Their peers and friend's kids who are addicted to YouTube, TikTok and other socials struggle reading. It's so very obvious to anyone who has kids at this age these days and I think a lot of parents are now realizing their mistakes and are often times embarrassed when they see these kids side-by-side in public displays of their work.
I keep seeing this over and over. I am on a parent technology group at my kids school and it is a monumental effort to keep these "technology directors" in check. They are some of the easiest to sell to given they go into conversations with companies like Ello with nothing but trust. They don't consider data, privacy and security of these students or very rarely. They often don't understand relationships with respect to technology and the areas where issues can surface or be present.
I hate everything about this blog post and, again, I truly hope Ello fails before they contribute to the continued degradation of these kids early lives.
> Teachers make mistakes, yes. But can you swap a teacher with another teacher without a student knowing? No, you can't. You can with a model and that is dangerous. And not only the company. If Ello wants to they can move students to a Chinese model and nobody would know the difference. Or the OAI model they've chosen can be dilluted or changed without Ello even knowing.
Sounds like this issue could be solved by an open-source teaching harness.
If anything, an app should be made where a child learns to correct an LLM's mistakes and learn that it isn't trustworthy.
Actually, better, don't put an LLM in front of children. At all.
EDIT: If a use case is for children who can't afford good education, then use an LLM to make educational materials for children, review them, and make them available for free. After all, the contents are ripped off from human educators anyway.