Now, the second project I’ll tell you about grew out of the Telegarden. As it was operating, my students and I were very interested in how people were interacting with each other, and what they were doing with the garden. So we started thinking: what if the robot could leave the garden and go out into some other interesting environment? Like, for example, what if it could go to a dinner party at the White House?
So, because we were interested more in the system design and the user interface than in the hardware, we decided that, rather than have a robot replace the human to go to the party, we’d have a human replace the robot. We called it the Tele-Actor. We got a human, someone who’s very outgoing and gregarious, and she was outfitted with a helmet with various equipment, cameras and microphones, and then a backpack with wireless Internet connection. And the idea was that she could go into a remote and interesting environment, and then over the Internet, people could experience what she was experiencing.
So they could see what she was seeing, but then, more importantly, they could participate, by interacting with each other and coming up with ideas about what she should do next and where she should go, and then conveying those to the Tele-Actor. So we got a chance to take the Tele-Actor to the Webby Awards in San Francisco. And that year, Sam Donaldson was the host. Just before the curtain went up, I had about 30 seconds to explain to Mr Donaldson what we were going to do.
And I said, “The Tele-Actor is going to be joining you onstage. This is a new experimental project, and people are watching her on their screens, there’s cameras involved and there’s microphones and she’s got an earbud in her ear, and people over the network are giving her advice about what to do next.”
And he said, “Wait a second. That’s what I do.” So he loved the concept, and when the Tele-Actor walked onstage, she walked right up to him, and she gave him a big kiss right on the lips. We were totally surprised — we had no idea that would happen. And he was great, he just gave her a big hug in return, and it worked out great.
But that night, as we were packing up, I asked the Tele-Actor, how did the Tele-Directors decide that they would give a kiss to Sam Donaldson? And she said they hadn’t. She said, when she was just about to walk onstage, the Tele-Directors still were trying to agree on what to do, and so she just walked onstage and did what felt most natural. So, the success of the Tele-Actor that night was due to the fact that she was a wonderful actor. She knew when to trust her instincts. And so that project taught me another lesson about life, which is that, when in doubt, improvise.
Now, the third project grew out of my experience when my father was in the hospital. He was undergoing a treatment — chemotherapy treatments — and there’s a related treatment called brachytherapy, where tiny, radioactive seeds are placed into the body to treat cancerous tumors. And the way it’s done, as you can see here, is that surgeons insert needles into the body to deliver the seeds.
And all these needles are inserted in parallel. So it’s very common that some of the needles penetrate sensitive organs. And as a result, the needles damage these organs, cause damage, which leads to trauma and side effects. So my students and I wondered: what if we could modify the system, so that the needles could come in at different angles? So we simulated this; we developed some optimization algorithms and we simulated this. And we were able to show that we are able to avoid the delicate organs, and yet still achieve the coverage of the tumors with the radiation.
So now, we’re working with doctors at UCSF and engineers at Johns Hopkins, and we’re building a robot that has a number of — it’s a specialized design with different joints that can allow the needles to come in at an infinite variety of angles. And as you can see here, they can avoid delicate organs and still reach the targets they’re aiming for. So, by questioning this assumption that all the needles have to be parallel, this project also taught me an important lesson: When in doubt, when your path is blocked, pivot.
And the last project also has to do with medical robotics. And this is something that’s grown out of a system called the da Vinci surgical robot. And this is a commercially available device. It’s being used in over 2,000 hospitals around the world. The idea is it allows the surgeon to operate comfortably in his own coordinate frame. Many of the subtasks in surgery are very routine and tedious, like suturing, and currently, all of these are performed under the specific and immediate control of the surgeon. So the surgeon becomes fatigued over time.
And we’ve been wondering, what if we could program the robot to perform some of these subtasks, and thereby free the surgeon to focus on the more complicated parts of the surgery, and also cut down on the time that the surgery would take if we could get the robot to do them a little bit faster?
Now, it’s hard to program a robot to do delicate things like this. But it turns out my colleague Pieter Abbeel, who’s here at Berkeley, has developed a new set of techniques for teaching robots from example. So he’s gotten robots to fly helicopters, do incredibly interesting, beautiful acrobatics, by watching human experts fly them. So we got one of these robots. We started working with Pieter and his students.