So today we’re going to start building this platform together, and we’re going to make the camera the first mainstream augmented reality platform. So if you take one thing away from today, this is it — right here: We’re making the camera the first augmented reality platform. So for those of you want to just roll out cameras across all our apps and you wonder what we might have been doing, that was Act 1. This is Act 2: giving developers the power to build for augmented reality in the first augmented reality platform: the camera.
All right. Let’s take a look at what this is going to look like. All right. So you’re going to be able to swipe to the camera and you’re going to start discovering effects that your friends are using and that are relevant to the place you’re at nearby. And you’re going to be able to scroll through all the effects and we have a lot of them.
Now we’re going to start today with all the basic effects that you’re used to: face masks, art frame, style transfers. Now since this is an open platform, you’re going to build to create your own and instead of having maybe ten or twenty options to choose from, you’re going to have thousands of options from creators all over the world from all different kinds of cultures and backgrounds and styles. And this is launching in Beta today. Now this is the first step, though. So we have a lot crazier stuff that I want to show you that’s going to be coming soon.
So now for real augmented reality, you don’t just want the ability to do those tools. You also want the ability to have realistic 3D objects and in order to do that, you need to have a platform that has — that gives them precise location, a realistic relationship with objects around them in their environment. So there’s an AI technique for doing this, called simultaneous localization and mapping, or SLAM, for those of you in the AI community.
And here’s how this works: so you’re going to build to easily create anything you want; you can write a fun message next to your breakfast and you can build a slot on to the table. And since we understand the depth of the table, you saw it was a clue, did in the right place as it came up and you’re going to build to pan around and it’s going to maintain its position on the table exactly as if it were a real object in the world. So we make this more fun, we’ll add some breakfast sharks swirling around my ball, some clouds, and you know, there you go. It’s got the depth right, so when they go behind the ball, that they’re included, it gets the depth of the table and all that.
All right. So there’s some pretty bulk AI work to make this all work and we can do this on a phone. So now — but we’re just going to start, this is the first one that I want to show you. These are the technological foundations for advanced AI. Let’s go to the next one.
Now since we’re mapping out these scenes in 3D, right? Since we have the depth of an image, we can go from taking a still photo to mapping out a whole 3D scene. So this actually was taken from a 2D still image in our office in Seattle. And from the still photo we constructed a 3D scene. And now because it’s a 3D scene we can pan around. How crazy is that? Crazy. All right. We can change the lighting, we can turn the lighting down, we can move the lighting from the front of the room to the back. And you can add all kinds of effects — we can fill the room up with water if we want; again it’s got the depth right, you can add a lot of bouncy balls – we’re fond of bouncy of balls. And we could fill the room up with skittles, because the future is delicious. I can’t help it.
All right, Now, look, we also have some of the best computer vision and object recognition work in the world, so that’s going to help you identify different things in the scene to help you surface relevant effects that you want to check out. So you’re going to build the top on the coffee mug and we’re going to surface effects that are relevant to coffee, you can have steam. You can add a second coffee mug, so it looks like you’re not drinking — you know, you’re not having breakfast alone. And since it’s a digital object, you can you make it bigger if you want or smaller; you can make it any size you want. You can top on a plant. You can add flowers that are blooming; you can water the plant with a raincloud or whatever else it is that you do with plants I guess. You can tap on the wine bottle and you can add an information card, where that shows what the vintage of the wine is, and you know what the rating is, and maybe where to get it or maybe in the future you’re going to like to buy it. So some of these effects are going to be fun and others are going to be useful.