Children outpace AI in language learning

When Michael C. Frank, a cognitive scientist at Stanford, declared that large language models have made ‘amazing’ progress, he was pointing to a yawning divide between artificial systems and human children. For decades, the only known entity that could learn a human language to perfect fluency was a child raised in a home. The arrival of ChatGPT four years ago gave the impression that machines could converse as naturally as people, yet beneath the fluency lies a stark data inefficiency. Children acquire language from a modest stream of speech, while today’s biggest models consume astronomical amounts of text. This contrast forms the core of a recent Technology Review feature that investigates why children still outperform the most sophisticated AI ever built.

The magnitude of the difference is difficult to grasp without numbers. A preteen raised in a linguistically rich household may have heard roughly 100 million words, and adding literacy by age 20 can push that total to about 300 million words. In contrast, Meta’s Llama 3.1, released two years ago, was pretrained on 15 trillion tokens, word‑like chunks that dwarf the child’s exposure many times over. Ethan Gotlieb Wilcox of Georgetown University notes that frontier models could be trained on ten times more data, but the internet’s supply of easily accessible text is finite. The article likens the LLM’s data diet to a stack of paper that would reach past the International Space Station, while the child’s 100 million words would form a pile just 20 metres high.

For the past decade, language models have mostly improved by scaling up parameters and ingesting ever larger corpora. Llama 3.1’s 15 trillion‑token pretraining run exemplifies this strategy, and industry analysts expect future models to require even more data. Yet experts warn that the well of publicly available text may run dry as early as the 2030s, creating a bottleneck for continued performance gains. The scarcity of training material forces researchers to reconsider whether sheer volume is the only path forward, or whether insights from human learning could unlock more efficient alternatives.

In response, a growing number of labs have turned to the BabyLM competition, an annual challenge organised by Warstadt, Choshen and other researchers. The competition asks teams to train language models on a ‘developmentally plausible’ corpus of just 100 million words (or 10 million for a toddler track) drawn from storybooks, dialogue, movie subtitles, Simple English Wikipedia, normal Wikipedia and actual child‑directed speech. The 2024 champion, GPT‑BERT, combined a next‑token prediction objective with a masked‑model loss, and when pretrained on roughly 100 million words it outperformed Meta’s Llama 2 70B, a model trained on about 15 trillion tokens, on a grammar benchmark. The result suggests that architectural tricks and objective design can partially compensate for limited data, though GPT‑BERT still falls far short of commercial LLMs in overall fluency.

Organisers also tested curriculum learning, which presents simple data before moving to more complex inputs, mirroring the presumed progression from baby talk to adult language. Aaron Mueller of Boston University, a BabyLM organiser, admits the approach ‘seems to really line up with ways that we believe humans are learning’, but the transformers in the competition did not benefit significantly from such ordering. Warstadt observed that the models learned effectively regardless of data schedule, indicating that the ordering hypothesis may not be essential for these artificial networks. The failure of curriculum learning to boost performance highlights how little is still known about aligning artificial training regimes with the subtle cues that guide child development.

Beyond text‑only experiments, researchers such as Brenden Lake have begun using head‑camera footage from the SAYCam project, which recorded two hours a week of three infants’ lives from six months to two-and‑a‑half years. The model Lake’s team built on 61 hours of raw video learned to associate visual objects with spoken words without pre‑programmed biases about word‑object mapping. This work suggests that multimodal exposure, seeing the referent while hearing the label, could compress the data needed for language acquisition.

Looking ahead, the central question remains whether AI can ever match the data efficiency of a human child. The poverty‑of‑the‑stimulus debate, originally championed by Noam Chomsky, still frames discussions about innate linguistic knowledge versus pure statistical learning. If models can be built to leverage limited, structured data, whether through architectural priors, multimodal inputs, or curriculum‑aware training, then the field may see chatbots that serve minority language communities or video‑based assistants that require far less footage. What to watch in the coming years includes the expansion of BabyLM to new languages, the emergence of head‑cam‑derived datasets, and any breakthroughs that claim to close the data gap without resorting to ever‑larger corpora.