And so they had a simple setup. They put a smartphone next to a keyboard, and they had people type, and then their goal was to use the vibrations that were created by typing to measure the change in the accelerometer reading to determine what the person had been typing.
Now, when they tried this on an iPhone 3GS, this is a graph of the perturbations that were created by the typing, and you can see that it’s very difficult to tell when somebody was typing or what they were typing. But the iPhone 4 greatly improved the accelerometer, and so the same measurement produced this graph. Now that gave you a lot of information while someone was typing, and what they did then is used advanced artificial intelligence techniques called machine learning to have a training phase. And so they got most likely grad students to type in a whole lot of things, and to learn, to have the system use the machine learning tools that were available to learn what it is that the people were typing and to match that up with the measurements in the accelerometer.
And then there’s the attack phase, where you get somebody to type something in, you don’t know what it was, but you use your model that you created in the training phase to figure out what they were typing. They had pretty good success. This is an article from the USA Today. They typed in: “The Illinois Supreme Court has ruled that Rahm Emanuel is eligible to run for Mayor of Chicago” — see, I tied it in to the last talk — “and ordered him to stay on the ballot.”
Now, the system is interesting, because it produced “Illinois Supreme” and then it wasn’t sure. The model produced a bunch of options, and this is the beauty of some of the AI techniques, is that computers are good at some things, humans are good at other things, take the best of both and let the humans solve this one. Don’t waste computer cycles. A human’s not going to think it’s the ‘Supreme might,’ it’s the Supreme Court, right? And so, together were able to reproduce typing simply by measuring the accelerometer.
Why does this matter? Well, in the Android platform, for example, the developers have a manifest where every device on there, the microphone, et cetera, has to register if you’re going to use it so that hackers can’t take over it, but nobody controls the accelerometer.
So what’s the point? You can leave your iPhone next to someone’s keyboard, and just leave the room, and then later recover what they did, even without using the microphone. If someone is able to put malware on your iPhone, they could then maybe get the typing that you do whenever you put your iPhone next to your keyboard.
There are several other notable attacks that unfortunately I don’t have time to go into, but the one that I wanted to point out was a group from the University of Michigan which was able to take voting machines, the Sequoia AVC Edge DREs that were going to be used in New Jersey in the election that were left in a hallway, and put Pac-Man on it. So they ran the Pac-Man game.
What does this all mean? Well, I think that society tends to adopt technology really quickly. I love the next coolest gadget. But it’s very important, and these researchers are showing, that the developers of these things need to take security into account from the very beginning, and need to realize that they may have a threat model, but the attackers may not be nice enough to limit themselves to that threat model, and so you need to think outside of the box.
What we can do is be aware that devices can be compromised, and anything that has software in it is going to be vulnerable. It’s going to have bugs.
Thank you very much.