Here is the full transcript of neuroscientist Stefano Baldassi’s TEDx Talk: The Renaissance of the Third Dimension at TEDxLA conference. This event occurred on December 3, 2016.
Stefano Baldassi – Neuroscientist
It says a lot that in a world filled with so much technology, we still rely on our Post-it notes. Even with phones, tablets, wearables, we still scramble to jot down our thoughts and post them all over desks, whiteboards, and sometimes even the very devices that they were supposed to replace in the first place.
As I was preparing for this very talk, I was juggling voice memos recorded from my morning run, files all over my laptop – texts, images, personal notes, you name it. Each day, this tangle of information got bigger; a new train of thought will generate a whole new dimension of ideas, running totally perpendicular to one another. Like a tree that kept growing new branches at bizarre angles.
Human creativity is a beautiful kind of chaos. We should support it, even when it’s messy. And yet, every device we use can only seem to show us flat, rectangular glimpses of it. It doesn’t take a neuroscientist to realize that this is not how humans evolved to think. Instead, imagine a world where technology is expected to accommodate the way we naturally think and act. In this world, our ideas would be organized in three-dimensional space, surrounding us with the content we need for our tasks and letting us reach right into it without clumsy middlemen like keyboards and screens. This would get us away from our desks and finally look up from our screens.
Our relationship with information would become a natural part of our physical lives rather than a separation from it. Instead of putting ourselves in a digital world, we could bring its best features in the real world we already live in. And this would be more than just the next thing. Our thinking has transitioned from two to three dimensions before. In fact, this transition was the singular innovation at the heart of the period we now call the Renaissance.
Since the first forms of human graphic expression, all the way to the Middle Ages, art was trapped in a crude, flat world. Well, this changed in Florence in the 15th century when the architect Filippo Brunelleschi opened the third dimension in art by being the first one to use linear perspective. From Brunelleschi onward, painted imagery began to feel less like an abstraction of our world and more like an extension of it. And this transition from flat to three-dimensional was more than just an upgrade for artists. It was a revolution in human ingenuity that extended far beyond painting for centuries.
For example, Galileo relied on his perspective drawings to discover properties of the Moon observed through the telescope, providing a key spark to ignite the Copernican revolution. Well, let me argue, today’s computers are due for a renaissance of their own. For years, I reached my lab in Florence by walking through the Duomo, the Baptistery, Piazza della Signoria, and across the Arno River. Each morning, I would walk by the work of those very artists, who opened the third dimension to the world in the Renaissance.
But in the last three years, I’ve been absorbing the new Florence, Silicon Valley, by participating to a new journey into the third dimension through a new technology called Augmented Reality. In augmented reality, the interface is no longer a flat screen but holograms projected into the world around us. We can reach out and touch them with our hands, we can attach them to real objects and access a myriad of information, or we can create new holograms, no longer Post-its, straight from our ideas and test in the blink of an eye how that fits in the real world, with our partners in crime.
But, wait a second, what’s neuroscience got to do with all this? Well, with augmented reality, we’re moving computers from desks or palms to our eyes, our senses, our brain. Our computers will no longer be mere objects in our space; they will create the very space in which we live and work. So, they’ll have to fit to our brains like a glove to a hand.
So, let me give you a few examples of how my team and I have been using neuroscience to accomplish this mission. We all get notifications of all sorts, don’t we? Well, in augmented reality, notifications can not only be distracting, but they can also be dangerous. Neuroscience of attention comes of great help here. Our brain has a circuit that ensures our continuous focus on our tasks. It’s the areas in blue in this figure.
When sudden, unexpected stimuli occur, the circuit highlighted in yellow takes over and breaks our flow on our task to redirect attention to this new event. Except, in our modern life, this circuit-breaking stimulus is no longer a tiger jumping at us in the jungle but our next Black Friday extended sale notification in our phone.
Another fundamental line of study comes from the observation that as much as one third of our brain processes visual information. But today’s computers are using only half of this potential. You see, our sense of sight consists of two distinct pathways.
The first is the ventral pathway, it answers the “what” questions by recognizing familiar objects, like faces, words, or objects, like this chair. But equally important is the dorsal pathway, which answers the “where” and “how” questions. If the ventral pathway tells us what we are looking at, the dorsal pathway helps us to understand the larger scheme of things and how we fit into it. The dorsal pathway is extremely sensitive to movement and closely tracks the space around us, so it plays a vital role in our actions, from intercepting a volleyball to the fine movements of an artist. That’s why today’s computers so easily frustrate us. They’re essentially designed for the ventral pathway only.
Instead, augmented reality uses space and depth to arrange information in a physically immersive way. It exploits both pathways and puts much more of our perception to use. To make it more practical, take a look at this photo of my friend Jerry Grant, Chief of Brain Surgery at Stanford Children’s Hospital. It shows only six of the 12 scattered displays featured in his operating room.
You may agree with me and with him that maintaining eye contact on the brain he’s operating on may in some cases make the difference between life and death. Instead, those surgeons live in a constant “texting-while-driving” type of experience, risking new errors with every distraction they get, no matter how important the information they pick up from those displays is. How wonderful would it be if you could eliminate those monitors and arrange them around the patient’s body. And more, align 3D scans right on the patient’s head, providing the ability to access any level of depth within the image to target the tools without mistakes. This would exploit the attentional system’s and the dorsal system’s sense of space, optimally.
But the ability to align the 3D scan right on the patient’s head is deeper than we think. In fact, just as the brain prefers to think spatially, instead of in two dimensions, our instinct compels us to reach right into our work rather than controlling it from a distance. Our brain understands the things within reach much better than those beyond it and has specific mechanisms to relate the things we see with the actions we take. And this is clearly another dimension of our brain that today’s technology doesn’t quite understand. But the brain has even more surprises.
Who’s heard of the mirror neurons? Well, as I was spoon-feeding my kids – over there – years ago, I couldn’t help opening my mouth as they were opening theirs. You know that. Those are neurons that activate when we make an action ourselves or when we see someone else doing the same thing. It’s as if our brain evolved with collaboration in mind, learning by example whenever possible. Flat computers block this powerful form of connection between us, while augmented reality restores it.
Jerry’s team of surgeons may connect with their hands and their tools before connecting with their voice. This was just a glimpse of the research my team and I have been doing over the last three years to help neuroscience humanize the future of computing. Imagine where we can be just a few years from now, wearing glasses not much different from those I’m wearing now, only smarter. Put your hand in front of you, and imagine you are holding a leaf. Then, imagine a hologram popping up from the leaf, revealing its secrets down to the cellular level.