Method may lead to new diagnosis, treatments for speech, language disorders
NICHD Research Conversations are audio interviews with NICHD scientists and grantees, on the latest NICHD-supported scientific findings. Listen to this Research Conversation at http://www.nichd.nih.gov/news/releases/Documents/NICHD_Research_Conversation_072613.mp3 (MP3 - 1.5 MB).
Rebecca Lazeration: Welcome to the National Institute of Health. I'm Rebecca Lazeration in the NIH's Eunice Kennedy Shriver National Institute of Child Health and Human Development, the NICHD. Thank you for joining us for today's Research Conversation.
Our host is Dr. Brett Miller of the NICHD Child Development and Behavior Branch. Dr. Miller will interview Dr. Tom Mitchell, Head of the Machine Learning Department at Carnegie Mellon University in Pittsburgh, and Dr. Marcel Just, Director of the Center for Cognitive Brain Imaging at Carnegie Mellon University, about applying the principles of machine learning to understand how the brain understands words.
Dr. Brett Miller: Thank you. Machine learning is a design and application of computer programs and systems that can learn from data. One relatively straightforward example of machine learning that most people have heard of is programming a computer to recognize sales or marketing terms in email messages so that the computer can block these spam messages and keep them from entering the user's mailboxes.
Dr. Tom Mitchell and Dr. Marcel Just's work involves applying broader machine learning principles to analyzing data from magnetic resonance imaging scans of the brain. These functional magnetic resonance imaging scans depict patterns of brain activity, and Dr. Mitchell and Dr. Just's work involves teaching computer systems to analyze these patterns of brain activity in order to decipher a word or a group of words that a person has just heard.
Tom, thank you so much for joining us today.
Dr. Tom Mitchell: My pleasure.
Dr. Miller: First, can you tell us why would we want a computer to be able to discern words based on the brain activity patterns?
Dr. Mitchell: Well, it's actually far beyond words. The thing we're interested in, in fact, is: how does the brain use neural activity to represent ideas, concepts? And words are one way that we use to get each other to think about particular concepts, but so are pictures and emotions and lots of ways to get at it.
But we use words because that's a good place to start. In fact, we used -- when we began this research -- we focused on nouns in particular and concrete nouns, meaning things that you could touch and feel like hammer, house, rock, car.
And the idea is that we'd like to understand how the brain represents ideas or words in this case, this special case, in terms of neural activity because if we could understand that, it would open the window to just looking at a lot of phenomena that go on in the brain. And if we could decode individual words or thoughts that were occurring in a sequence of brain activity, it would then just give us a whole new insight into how the brain operates.
Dr. Miller: Great. What do you believe the applications will be for this type of research?
Dr. Mitchell: The applications could range from trying to understand and make better treatments for language disorders. Imagine for example somebody with dyslexia who may have some trouble reading. If we could understand by looking at their neural activity how dyslexics differ from standard readers, it might give us a starting point for coming up with ways to help these people.
You could imagine also education, if you're learning a second language. I know when I try to learn a second language, one of my problems is I can't understand the language very well. In fact, when I hear it, I'm not even sure which words in the sentence I'm hearing. Sometimes I can't tell where the words end and the next one begins. But I bet if we understood how to look at my neural activity while I'm trying to process that other language, you could help me understand which parts of the sentence I got right and wrong, and maybe then give me a better training session to learn that language.
Dr. Miller: Marcel, thank you for joining us as well.
Dr. Marcel Just: My pleasure.
Dr. Miller: Can you explain how you go about analyzing the responses to the speech in the brain?
Dr. Just: Yes. And here's the secret sauce. Thoughts or representation of words or speech or whatever isn't done by any one particular place in the brain. It's always a set of a team played by several, five, ten, fifteen brain areas working in conjunction and collaboration with each other.
So we don't look for the place in the brain where a particular idea is represented, but the many places in the brain where various facets of the idea is represented. And the reason this occurs is because our brain is made up of many parts. It's evolved over millennia, maybe millions of years, and various parts have been gradually added and evolved and so on, and now in its full glory, all of these parts join in in order to represent something as simple as an apple or a hammer or so on.
And so in answer to your question as to how we analyze it, we find those various disparate separate players in the brain that come to play when it comes time to think about a hammer or an apple. And what we do is look for multiple places in the brain that together respond when an apple or a hammer is being thought about. And so that's the secret sauce, looking for multiple places in the brain that together are involved in a particular form of thought.
Dr. Miller: Great. How much do your findings advance our understanding of human psychology and communication?
Dr. Just: I think that the importance of these findings is that they reveal the building blocks of thought. And the building blocks aren't the same as the building blocks of the brain. I refer to the various brain areas that come to play, but those aren't really the building blocks of thought. It's the kind of the configurations that they together form.
To give an analogy to a sports team, it's like say a passing play in football involves a quarterback and a receiver and blockers and so on. And so it's not just the players but what they do. And so you could say the building blocks of a football team are the various kinds of plays that the coach has in his playbook.
The building blocks of thought are the various ways, the various components of thought that we have. We have, to give you an example, when we're talking about concrete objects like apples and hammers and so on, one of the components of that thought is how your body interacts with it. How you hold a hammer versus how you hold an apple versus how you kick a soccer ball.
So one of the dimensions of human thought about concrete objects is how your body's going to interact with it. And that'll distinguish a chair from a banana and so on, or a pair of pliers.
Another one is can you let the object cover you or envelope you or provide you with shelter? So one of the components of thought involving a set of brain areas analyzes whether a particular object provides shelter. Now an obvious object is a house, but maybe also a desk could provide shelter if you could climb under, maybe a closet, maybe a tent, maybe a cave.
So those are two examples of the components of how you might think about a concrete object. So I think that the big contribution is analyzing out the components of thought. I've given you two examples of the components of physical objects, but we obviously think about much more than physical objects. We think about -- we have emotions, we think about social interactions, we think about quantities. It's whatever the human mind is capable of thinking.
And whenever we think, there are these components, these basic building blocks. Not anatomical building blocks, but conceptual building blocks. And that's what we're finding.
Dr. Miller: Great. Thank you so much. So that concludes our Research Conversations for today. I'd like to thank Drs. Mitchell and Just again for taking their time out to speak with us. And I'd like to thank our listeners for joining us as well. Thank you.
###
About the Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD): The NICHD sponsors research on development, before and after birth; maternal, child, and family health; reproductive biology and population issues; and medical rehabilitation. For more information, visit the Institute's website at http://www.nichd.nih.gov/.
BACK TO TOP