Better Understanding Language Processing In The Human Brain While Furthering AI
Meta AI has released the culmination of recent research into deep language processing and what they've looked into, towards bridging the gap between language processing in the human brain and AI.
Created on April 28|Last edited on April 28
Comment
Meta AI has been putting a lot of research into lanuage learning recently, going so far as to employ the use of brain scanning technology to see how a human brain lights up where experiencing language. In the aim to bridge the gap between human language processing and ai language processing, Meta AI has worked to expand and develop datasets required to meet that goal, and are even discovering the shocking similarities between how human brains and machine learning models process language.
In a new post on the Meta AI blog posted today, researchers explain what they've discovered about language processing in machine learning and the human brain in the studies they've been conducting recently, and continue to explore.
Language processing in machine learning vs the human brain
We still don't know what mechanism in the human brain allows us to learn language so easily. As children, the rate at which we pick up language is incredible, something absolutely unmatched compared to today's cutting edge language processing models. Language processing models are rigorously trained on billions of sentences, while humans are able to achieve much better results with only a few million sentences and are even able to quickly learn new concepts to add into their repitoire.
In analyzing a large dataset of brain scans collected with both fMRI and MEG methods and comparing them with various deep language models, the researchers observed that the better the models can predict words, the more they resembled the electrical signals of a human brain in simple language processing tasks.
The importance of prediction in language processing
Given the similarity in activity between the human brain and deep language models in word prediction, a focus was put on how we can understand how the two work compared to each other.
While deep language models are generally easily able to predict upcoming single words, the human brain does much more than that. An example in the post is the phrase "Once upon a ..."; Surely reading that text just now conjured images of castles, princesses, knights, and dragons (or if not medieval adventure, some other story book setting) simultaneously with the word that completes the phrase, "time". The human brain has the ability to extrapolate this simple 3 word prompt into a whole display of ideas, yet the best artificial language processing models just simply insert the word "time".
The gap between human brains and AI is decreasing
With new understanding into the workings of human brains and AI, we simulatenously move both fields forwards, each jumping off of new discoveries from the other. As we develop models capable of more human-like ability, we are tipped off about ways the human brain may be working, and from there we can refine and expand our models and so on. With continued research, we may one day approach truly human-like AI.
Find out more
Add a comment
Tags: ML News
Iterate on AI agents and models faster. Try Weights & Biases today.