Instead, augmented reality uses space and depth to arrange information in a physically immersive way. It exploits both pathways and puts much more of our perception to use. To make it more practical, take a look at this photo of my friend Jerry Grant, Chief of Brain Surgery at Stanford Children’s Hospital. It shows only six of the 12 scattered displays featured in his operating room.
You may agree with me and with him that maintaining eye contact on the brain he’s operating on may in some cases make the difference between life and death. Instead, those surgeons live in a constant “texting-while-driving” type of experience, risking new errors with every distraction they get, no matter how important the information they pick up from those displays is. How wonderful would it be if you could eliminate those monitors and arrange them around the patient’s body. And more, align 3D scans right on the patient’s head, providing the ability to access any level of depth within the image to target the tools without mistakes. This would exploit the attentional system’s and the dorsal system’s sense of space, optimally.
But the ability to align the 3D scan right on the patient’s head is deeper than we think. In fact, just as the brain prefers to think spatially, instead of in two dimensions, our instinct compels us to reach right into our work rather than controlling it from a distance. Our brain understands the things within reach much better than those beyond it and has specific mechanisms to relate the things we see with the actions we take. And this is clearly another dimension of our brain that today’s technology doesn’t quite understand. But the brain has even more surprises.
Who’s heard of the mirror neurons? Well, as I was spoon-feeding my kids – over there – years ago, I couldn’t help opening my mouth as they were opening theirs. You know that. Those are neurons that activate when we make an action ourselves or when we see someone else doing the same thing. It’s as if our brain evolved with collaboration in mind, learning by example whenever possible. Flat computers block this powerful form of connection between us, while augmented reality restores it.
Jerry’s team of surgeons may connect with their hands and their tools before connecting with their voice. This was just a glimpse of the research my team and I have been doing over the last three years to help neuroscience humanize the future of computing. Imagine where we can be just a few years from now, wearing glasses not much different from those I’m wearing now, only smarter. Put your hand in front of you, and imagine you are holding a leaf. Then, imagine a hologram popping up from the leaf, revealing its secrets down to the cellular level.
Then you can use your hand to move water inside the leaf vein and observe the minerals that nourish the plant and promote photosynthesis. And you can do that with your friends or your kids. It reminds of this photo of my own father reading to my daughter, years ago. The technology has changed, but the idea is the same, people connecting in real ways, creatively and personally. By understanding how our brain works, we can create computers that deliver these experiences, extending our cognitive abilities, instead of trying to replace them.
Our creativity will finally be able to run free without compromises and without limits. In short, it will be a new renaissance. Thank you.