Here is the full text of AI researcher Rosalind Picard’s talk titled “Brain Activity Revealed Through Your Skin: Stress, Sleep, & Seizures” at TEDxNatick conference.
Rosalind Picard is an American scholar who is Professor of Media Arts and Sciences at MIT, founder and director of the Affective Computing Research Group at the MIT Media Lab, and co-founder of the startups Affectiva and Empatica.
This story begins when my students at MIT and I were building new technology to help people understand the emotions of others. We were working with people on the autism spectrum like this little boy here, and we created technology to read facial expressions, which you can get in your phone today thanks to the work of Affectiva.
But while we were creating this, and I was talking with one of my friends with autism, who actually talked by typing, she said to me, “Roz, you’ve got it wrong. Our biggest problem is not understanding other people’s emotions, it’s you understanding our emotions.”
And at first I thought, “Oh dear, it’s me.” And indeed, I have room to improve. But it turns out we all have room to improve.
And as I asked her, “Well, what are the emotions that we’re missing, that you most wish we could understand?”
She said, “There’s huge anxiety and stress that we’re experiencing. Many times, the environment, the lights, the sounds, the smells are driving us crazy, and we’re about to explode, and yet on the outside we may look just like we’re shutting down. So we’re being misread.”
And I realized we had built technology in our lab years before that measures something called electrodermal activity. That’s a big word that is referring to a general phenomenon whereby your skin becomes more conductive when you get more nervous, when your sympathetic nervous system, your autonomic – sounds like automatic, but it’s autonomic nervous system – that controls your heart beating, your lungs breathing, all of these parts of your body automatically.
When the sympathetic branch of that goes high in fight or flight or with general excitement, it can make your hands sweaty. And even when you don’t feel sweaty, it can make electrical changes that we can sense on the surface of the skin.
Now, this was traditionally done with wires and electrodes in the lab, and that was not very portable. At the Media Lab, we developed ways to measure it from the wrist, and the ankles, and other places on the body. The first time that I saw the skin conductance data 24/7 was this picture from an MIT student.
What you see here are seven days of data, 24 hours a day. If we zoom in, you’ll see that there are places where the skin conductance is going really high, where, again, there’s more excitement, or more emotional load, or more cognitive load. In this case we see MIT homework sets caused quite a lot of activation.
Now, to my utter surprise, sleep also caused a lot of activation, which was not what I expected. In fact, sleep is often the biggest peak of the day. And the reason for this is still a mystery. But some of the things you’re going to hear shortly, I think, are about to give us more insight into this mystery.
And also I have to admit, to the embarrassment of we MIT professors, the low point every day was classroom activity. Once we got the data continuously, not in the lab but in real life, we started learning all kinds of things, and there were a number of very exciting surprises, and I’m going to tell you the story of one of them.
And this one happened when a young man working as an undergrad in my lab came to me at the end of the semester, and he said, “Professor Picard, could I please borrow one of those sensors for my little brother? He has autism. He can’t speak. And I want to see what’s stressing him out.”
And I said, “Sure. In fact, don’t just take one, take two.” Because it was a long winter break, and they were hand built with wires hanging out back then, and they often broke.
So he takes the two sensors; he puts them on his little brother. And back at MIT, I go to my computer, and I look at the screen, and I see data like I just showed you for this little boy. This day looked pretty normal. This day looked pretty normal. I go to the next day, and my jaw drops.
He had put the sensors on the left and right wrist at the same time. Okay. And one of the wrists, the data had gone so high, that the sensor must be broken. We have stressed people out at MIT every way I can imagine. Qualifying exams. Public speaking. We have measured Boston driver stress – huge peaks. But nothing as big as what I saw on this little boy.
And the weird thing was it was on one side and not the other. The other side wasn’t responsive at all. And I thought, with my electrical engineering hat on: How can this happen? How can – unless the sensors are broken – how can you be excited on one side of your body and not the other? This just didn’t make sense.
After a lot of debugging that I won’t go into here, I finally gave up, and I did something I’ve never done before. I called a student at home on vacation: “Hi. How’s your Christmas going? How’s your little brother? Hey, any idea what happened to him.” And I gave him the exact date and time. And he said, “I don’t know, I’ll check the diary.”
Diary?! MIT student keeps a diary? Quick prayer. He comes back; he has the exact date and time. He checks it with me, and he says, “That was minutes before he had a grand mal seizure.”
Now, I didn’t know, really, what a seizure was. I started to do some research. Next thing I know, I’m on the phone with Dr Joe Madsen, head of neurosurgery at Children’s Hospital, Boston: “Hi. Dr Madsen. My name’s Dr Rosalind Picard. Could you tell me, is it possible that somebody could have a huge sympathetic nervous system surge many minutes before a grand mal seizure?”
And he says, “Probably not. But, you know, we’ve sometimes had patients who have hair stand on end on one arm before a seizure.”
I said, “On one arm?” And I told him that this didn’t just happen the way it usually happens, it happened on only one side. And he got interested, and we got Institutional Review Board approval, the ethics board at the hospital, we built a lot more sensors, we enrolled families fully consented, where they are bringing their children in not only to have around-the-clock monitoring of EEG for the brain waves, ECG for the heart, but now EDA, electrodermal activity, for the skin conductance.
And here’s an example of what we saw. This is data from a 17-year-old boy, and what you see is the skin conductance here. And in the middle is the sleep. Those are the biggest peaks I was showing you from the MIT student.
Well, those are like little baby maple trees next to the three other peaks that are like redwoods coming out of the data. The other three peaks are grand mal seizures. And we found 100% of grand mal seizures had a significant increase in the signal that we could measure on the wrist.
That signal in a grand mal is also accompanied by convulsive motion that you see under the three red lines in the little blue boxes. That shows a lot of accelerometer activity.
Now, most of the sensors on the market today that people with epilepsy could use only measure that movement, and that can be confused with brushing your teeth, strumming the guitar – you get a lot of false alarms.
But through this accidental finding, we’re able to combine it with the skin conductance and get a more accurate detector. And we also asked: Why are these so big and so long? The seizure’s only a couple of minutes. And this response is lasting much longer than a normal stress response. Was it that the person was convulsing so much that they got sweaty?
No, it turns out it’s completely unrelated to the length – or strength of the convulsions. But it turns out to be related to – it was really surprising to me – I learned that while most seizures – and here is brainwave activity from a seizure; on the left, you see it kind of going crazy; this is EEG traces from the scalp – most of them end and then the brain activity looks normal.
But here it’s going flat after the seizure. And that suppression happens – well, was observed, in 100% of the published cases where a patient happened to be wearing an EEG when they had a seizure, nobody was with them, and they passed away after the seizure.
Now, fortunately, death in epilepsy is rare. But it turns out it’s more common than death from SIDS, from AIDS, and from house fires in the US. It’s called Sudden Unexpected Death (SUDEP) in Epilepsy, and I bet most people here have never heard of it.
And yet, if you look at the statistics of neurological disorders, and here we see the years of lost life for people with stroke and ALS – Lou Gehrig’s disease – or multiple sclerosis or Alzheimer’s or Parkinson’s, SUDEP is number two on this chart. It needs to be better known.
As I learned more about seizures, I learned that they’re kind of like little electrical fires. You can start like a little brush fire. And it can stay localized in your brain, and it can simply cause an unusual sensory experience, or déjà vu.
You may not see anything outwardly that is going on differently when the seizure is happening inside. But the more dangerous seizures can make you unconscious and can cause these convulsions, and those are called the grand mal seizures.
Now, you wouldn’t build a house today without having a smoke detector. And yet, SUDEP kills more people in the US every year than house fires, and patients are sent home not only without some kind of equivalent of a smoke detector, but without even the ability to talk to a lot of people about what’s going on.
So that led to this short clip I’m going to show you next.
(Video clip): Rosalind Picard: Our device is designed to detect unexpected seizures. Embrace is designed to save lives for a lot of people with epilepsy. Seizures can seriously hurt or even kill people, and we need an alert to intervene. One in 26 people in the US will develop epilepsy at some point during their lifetime. Everybody with epilepsy should be able to have a device that alerts a friend or a family member to come help at the time that they might need it. – (Video ends)
Thanks to the teamwork of Empatica in partnership with the Epilepsy Foundation and the ability of new crowdsourcing sites like Indiegogo to reach the public, this project was fully funded. And, for the first time I’ve ever seen in crowdfunding of a cool new technology, not only were devices able to be gotten to people who could afford one, but through the partnership with the Epilepsy Foundation and some generous anonymous donors, we were able to give a device – for each one that was purchased – to a child and a family who could not afford a $200 device.
Now, I’ve told you a quick story here, but we’re not over yet. This story started with trying to help understand and communicate emotion better in an individual with autism and express the anxiety. Today, thanks to the hard work of Empatica, I’m able to wear a device that measures my autonomic stress.
It can gently vibrate and tell me that it’s going up, privately. Or, if I choose, I can have the signal sent to somebody who I care about, who I know I can trust. It reminds me of when the director of the Media Lab said years ago, “Roz, when are you going to build me the mood ring that tells me my wife’s mood before I go home?”
Well, now we can tell somebody a little bit more about the stress contributing to your mood. The same device can also now – has the potential to help people with autism communicate what’s going on even when they can’t do it verbally. And the device also has the ability to run an algorithm on board that can detect those unusual events that might be a grand mal seizure, and bring an alert so that somebody can come and check on you.
Because we’ve learned that when somebody is there, SUDEP is a lot less likely to happen. And the person who comes there may not have to do anything more than simply say the name, or turn the person over, or ask if they’re okay.
Simply touching or stimulating a person very gently after a grand mal seizure and helping them get in a safe position could help them take that next breath and help that seizure not be the terminal kind.
Now, as I was telling about this work to one of my friends, he said to me, “Oh, Roz, I have epilepsy.”
“What?! You have epilepsy? You’ve been my friend for 20 years and you’ve never told me you have epilepsy? Why didn’t you tell me?”
And he said, “Well, I just have felt uncomfortable telling anybody that I have seizures, and I don’t have them very frequently.”
Epilepsy is stigmatized, and that needs to change. And we can do that here today, we can begin that process of change. And to begin that process of change, I’m going to introduce you to somebody very special. First a word or two of background.
Imagine being 13 years old and lying on this surgical table with your head under that big lamp. Now, fortunately, the surgeon at the end is Dr Joe Madsen, who’s amazing. But he’s about to cut a big hole in your skull and go into your brain deep with sharp objects and reconfigure some things. I don’t know about you, but I would not want to volunteer for this. This is a very scary surgery.
And this is Bailey Dwyer on the right, who has just gone through this surgery and is recovering nicely. Bailey’s very brave, and she’s also very courageous and generous in another way. I know this looks like a teenager dressing up for Halloween as a zombie, right? It’s really cool to put on the fake blood and the gauze and all that stuff.
But this is the real thing. Bailey has wires coming out of her head here that go not just under the surface of the skin, they go deep into the brain, into the regions of the brain involved in memory, emotion, attention, and other vital processes.
And Bailey graciously volunteered to not only have what she needed to have done, but to participate in scientific experiments that enable all of us to benefit from and learn more about how these deep regions of the brain operate, how they map to other things we can measure on the surface of the body, and how we can use this information to help lots of other people.
People like Bailey are my heroes. And I want you to join me right now in welcoming to our stage Bailey Dwyer.
Bailey Dwyer: First, thank you, Roz, for introducing me. I’ll start by saying my name is Bailey and I’m a junior at Brookline High School. When I was in eighth grade, I had two brain surgeries for my epilepsy. I currently do softball and am studying for my SATs.
When I had my brain surgery, I was scared to tell anyone of why I would be away because I was worried about how they would stereotype me. It’s important to educate people on what epilepsy is and get rid of the stigma around it. By raising the issue and telling people what it is will not only raise awareness but save many lives. Thank you.
Rosalind Picard: Bailey, you drove here today too, right?
Bailey Dwyer: I did, yes.
Rosalind Picard: Yes! What does it say about our culture today when a young high school student cannot tell his or her friends about why they’re going away from school for a while to have major brain surgery? We can change that. You can change that. And all you have to do when you leave here today is chat about it, talk to people about it. Ask your friends if they have epilepsy.
Tell them what you learned about here today. In doing this, together, we can destigmatize this, we can help people get better treatment, and by doing this, you may also help to save a life. Thank you.
- How College Loans Exploit Students For Profit: Sajay Samuel (Transcript)
- How to Design for Dignity During Times of War: Slava Balbek (Transcript)
- The Next Outbreak? We’re Not Ready: Bill Gates (Transcript)
- Revealing The Lost Codex of Archimedes: William Noel (Transcript)
- The Power of Civil Momentum: Annamalai Kuppusamy (Transcript)