Here is the full text of AI researcher Rosalind Picard’s talk titled “Brain Activity Revealed Through Your Skin: Stress, Sleep, & Seizures” at TEDxNatick conference.
Rosalind Picard is an American scholar who is Professor of Media Arts and Sciences at MIT, founder and director of the Affective Computing Research Group at the MIT Media Lab, and co-founder of the startups Affectiva and Empatica.
Rosalind Picard – TEDx Talk TRANSCRIPT
This story begins when my students at MIT and I were building new technology to help people understand the emotions of others. We were working with people on the autism spectrum like this little boy here, and we created technology to read facial expressions, which you can get in your phone today thanks to the work of Affectiva.
But while we were creating this, and I was talking with one of my friends with autism, who actually talked by typing, she said to me, “Roz, you’ve got it wrong. Our biggest problem is not understanding other people’s emotions, it’s you understanding our emotions.”
And at first I thought, “Oh dear, it’s me.” And indeed, I have room to improve. But it turns out we all have room to improve.
And as I asked her, “Well, what are the emotions that we’re missing, that you most wish we could understand?”
She said, “There’s huge anxiety and stress that we’re experiencing. Many times, the environment, the lights, the sounds, the smells are driving us crazy, and we’re about to explode, and yet on the outside we may look just like we’re shutting down. So we’re being misread.”
And I realized we had built technology in our lab years before that measures something called electrodermal activity. That’s a big word that is referring to a general phenomenon whereby your skin becomes more conductive when you get more nervous, when your sympathetic nervous system, your autonomic – sounds like automatic, but it’s autonomic nervous system – that controls your heart beating, your lungs breathing, all of these parts of your body automatically.
When the sympathetic branch of that goes high in fight or flight or with general excitement, it can make your hands sweaty. And even when you don’t feel sweaty, it can make electrical changes that we can sense on the surface of the skin.
Now, this was traditionally done with wires and electrodes in the lab, and that was not very portable. At the Media Lab, we developed ways to measure it from the wrist, and the ankles, and other places on the body. The first time that I saw the skin conductance data 24/7 was this picture from an MIT student.
What you see here are seven days of data, 24 hours a day. If we zoom in, you’ll see that there are places where the skin conductance is going really high, where, again, there’s more excitement, or more emotional load, or more cognitive load. In this case we see MIT homework sets caused quite a lot of activation.
Now, to my utter surprise, sleep also caused a lot of activation, which was not what I expected. In fact, sleep is often the biggest peak of the day. And the reason for this is still a mystery. But some of the things you’re going to hear shortly, I think, are about to give us more insight into this mystery.
And also I have to admit, to the embarrassment of we MIT professors, the low point every day was classroom activity. Once we got the data continuously, not in the lab but in real life, we started learning all kinds of things, and there were a number of very exciting surprises, and I’m going to tell you the story of one of them.
And this one happened when a young man working as an undergrad in my lab came to me at the end of the semester, and he said, “Professor Picard, could I please borrow one of those sensors for my little brother? He has autism. He can’t speak. And I want to see what’s stressing him out.”
And I said, “Sure. In fact, don’t just take one, take two.” Because it was a long winter break, and they were hand built with wires hanging out back then, and they often broke.
So he takes the two sensors; he puts them on his little brother. And back at MIT, I go to my computer, and I look at the screen, and I see data like I just showed you for this little boy. This day looked pretty normal. This day looked pretty normal. I go to the next day, and my jaw drops.
He had put the sensors on the left and right wrist at the same time. Okay. And one of the wrists, the data had gone so high, that the sensor must be broken. We have stressed people out at MIT every way I can imagine. Qualifying exams. Public speaking. We have measured Boston driver stress – huge peaks. But nothing as big as what I saw on this little boy.
And the weird thing was it was on one side and not the other. The other side wasn’t responsive at all. And I thought, with my electrical engineering hat on: How can this happen? How can – unless the sensors are broken – how can you be excited on one side of your body and not the other? This just didn’t make sense.
After a lot of debugging that I won’t go into here, I finally gave up, and I did something I’ve never done before. I called a student at home on vacation: “Hi. How’s your Christmas going? How’s your little brother? Hey, any idea what happened to him.” And I gave him the exact date and time. And he said, “I don’t know, I’ll check the diary.”
Diary?! MIT student keeps a diary? Quick prayer. He comes back; he has the exact date and time. He checks it with me, and he says, “That was minutes before he had a grand mal seizure.”
Now, I didn’t know, really, what a seizure was. I started to do some research. Next thing I know, I’m on the phone with Dr Joe Madsen, head of neurosurgery at Children’s Hospital, Boston: “Hi. Dr Madsen. My name’s Dr Rosalind Picard. Could you tell me, is it possible that somebody could have a huge sympathetic nervous system surge many minutes before a grand mal seizure?”
And he says, “Probably not. But, you know, we’ve sometimes had patients who have hair stand on end on one arm before a seizure.”
I said, “On one arm?” And I told him that this didn’t just happen the way it usually happens, it happened on only one side. And he got interested, and we got Institutional Review Board approval, the ethics board at the hospital, we built a lot more sensors, we enrolled families fully consented, where they are bringing their children in not only to have around-the-clock monitoring of EEG for the brain waves, ECG for the heart, but now EDA, electrodermal activity, for the skin conductance.
And here’s an example of what we saw. This is data from a 17-year-old boy, and what you see is the skin conductance here. And in the middle is the sleep. Those are the biggest peaks I was showing you from the MIT student.
Well, those are like little baby maple trees next to the three other peaks that are like redwoods coming out of the data. The other three peaks are grand mal seizures. And we found 100% of grand mal seizures had a significant increase in the signal that we could measure on the wrist.
That signal in a grand mal is also accompanied by convulsive motion that you see under the three red lines in the little blue boxes. That shows a lot of accelerometer activity.
Now, most of the sensors on the market today that people with epilepsy could use only measure that movement, and that can be confused with brushing your teeth, strumming the guitar – you get a lot of false alarms.
But through this accidental finding, we’re able to combine it with the skin conductance and get a more accurate detector. And we also asked: Why are these so big and so long? The seizure’s only a couple of minutes. And this response is lasting much longer than a normal stress response. Was it that the person was convulsing so much that they got sweaty?