Meta’s Andrew Bosworth on moving Facebook to the metaverse

Last week, Facebook announced a major corporate rebrand by changing its company name to Meta. As CEO Mark Zuckerberg told The Verge, the new name is meant to solidify the social media giant’s long-term bet on building the metaverse.

Leading that work directly is Andrew Bosworth, a 15-year-veteran of Facebook who leads its Reality Labs division building consumer hardware and software, including the Quest VR headset. His organization has over 10,000 employees and is spending at least $10 billion this year alone. Before Reality Labs, he led the division building Facebook’s advertising business and co-invented staple features like the News Feed and Groups. Next year, he will become the chief technology officer of Meta, expanding his remit to include the company’s artificial intelligence and broader engineering teams.

Bosworth (who goes by “Boz”) talked with senior reporter Alex Heath about Facebook’s rebrand to Meta, how content moderation will work in the metaverse, and the hardware journey from virtual to mixed reality and, eventually, AR glasses. Bosworth also touched on the controversy surrounding the Facebook Papers, a trove of internal documents leaked by an ex-employee named Frances Haugen, that he argues “don’t tell a particularly objective story.”

This transcript has been lightly edited for clarity.

Andrew Bosworth, you are the VP of reality labs at Meta and soon-to-be CTO of Meta, welcome to Decoder.

Thanks for having me.

I want to get into everything you guys just announced at Connect: a big new corporate rebrand focused on the metaverse. But first, I have to know, what were the other name options? Because over the last week, I’m sure you’ve seen, there have been a lot of guesses. I personally would’ve loved the Washington Football Team, but I was really curious if you could just give a short list of what were the options?

This is probably someone else’s story to tell more than mine, especially Alex Schultz, who [was] pretty masterful, at the scale that this company operates, to have this held out as long as we did was, I think, pretty spectacular. Obviously, one of the names you see us investing a lot in is Horizon. And at some point you had to make a choice of, well, you want the entire company to be a metaverse company but you also want to have a product within that. So we talked about Worlds and Rooms and Home as kind of these three pillars of what Horizon is. So that was one of the ones that we were discussing. I know you took a shot in the dark on that one. It was a near miss.

I spoiled a little bit of your all’s surprise.

It was a good scoop, man. We don’t know how you did it. Don’t tell me. I don’t like having the surprise ruined for me.

I thought it might be Horizon. Meta was the other one on the short list.

You sent me a DM and you’re like, “It better not just be Meta.” I just didn’t respond. Reporters troll me all the time for information and you just can’t respond. I didn’t respond. And then today, the moment it was announced, the first thing I did was reply to that response.

Yeah, you did. And I should say you mentioned Alex Schultz earlier. For people who don’t know, that’s Facebook’s CMO [chief marketing officer], one of your—

Meta’s CMO, even.

That’s right, Meta’s CMO. This is going to be the whole podcast. Alex is a longtime colleague of yours. So you oversee a very large part of Meta now, and it’s going to get bigger when you become CTO [chief technology officer] next year. I’d love to get into all that. First, for the audience, I’d love for you to just quickly walk through your background, what you worked on before this, and how you got to be in this role?

I have a really wide-ranging background at Facebook. I’ve been at Facebook a long time. As an undergraduate, I studied artificial intelligence. I had a brief stint at Microsoft, actually, which was great, it taught me a lot about professional software development and management, but joined Facebook relatively early in my career. The first thing I did out of the gate, and what a great assignment, working with Chris Cox and Ruchi Sanghvi, was the News Feed. And I built all the AI, the ranking, and we built the first, as far as I know, the first ranked content feed, and I built all the rank and the AI behind it, Chris built the front end, and Ruchi was the PM.

I wanted to just stop here for a second. I don’t think people understand how early you were at Facebook. Do you have a sense of how many people there are that have been there longer than you?

I think now five or six that have been there longer than me at this point.

When I joined there were probably 15 of us, 15 engineers at the company. And a few of us are still here. Obviously, Zuck [Mark Zuckerberg], Naomi [Gleit], some of the big names. Chris Cox, I think we’re going to count it, he beat me by a month, although he did leave for a year. So we’re not entirely clear if that resets the clock or how it works. It was early and we were just a website for college kids. I remember when we were building the News Feed in January 2006, I asked Mark, “What’s the scale?” He was like, “Well, we just hit our 5 million user mark, build it for 10 million. That’ll take us a long time to get to.” Of course, by the time we launched six months later we’d blown past 10 million users.

It was a crazy journey. I really worked across all of our product. Back then, it was really homogenous. You worked on the entire product. You worked on photos and wall and profiles and all these things together. And then I had a big stint running communication stuff. So, Messenger is one of the teams that I was leading when we created Messenger. And Groups, I’m really proud of that one. It was a small team that I was leading that created what is now one of our, I think, great contributions, is Facebook Groups.

I worked on a lot of the teams there in the interim, privacy and infrastructure and site speed and these things, to mobile ads. And then ultimately to ads. And I ran the ads business from 2013, I suppose, until 2017, which was a dramatic period of growth for us. It was a fun time to be working on that product. And then, yeah, now the last four years, I’ve been working on, first, it was called AR / VR, then Facebook Reality Labs and now just Reality Labs, which is a part of Meta.

Gosh, so many name changes.

I’m like Daniel-san in The Karate Kid. I spent a lot of time painting the fence, spent a lot of time waxing the car. And now I’m, maybe, equipped to do the job that I’m in.

So now you have this org that is over 10,000 people now. Mark said on the last earnings call, you guys are going to spend at least $10 billion this year alone on the metaverse, on building all this stuff that you directly oversee. When you think back to being that early, being one of the first 15 engineers or so, did you ever think you’d be overseeing a team and a budget that large? I don’t think anyone really expected that Facebook would eventually be spending this much on this stuff.

No, I didn’t have a concept of that. I don’t think anyone can ever really have a concept of what it means to have a team that numbers so many phenomenally talented people who could get a job anywhere. They choose to work with us. We’re grateful for that. And the amount of money is hard to fathom. 

To some degree, I think all of us as just a species, we’re a little bit innumerate as it relates to big numbers. Big tech numbers we think of as being big, big dollars. And those pale in comparison to the government, but they’re certainly bigger than anything that we deal with in our day-to-day lives. And so, I do think this is one of the pieces for me that maybe I feel like we’re at a privileged point in history, because the magnitude of technological shift that we are trying to manifest here hasn’t been attempted in a long time.

I think certainly the creation of the internet in the late ’80s, early ’90s, mid-90s, was one of them. I think if you go back to Xerox PARC and the work they did on the Alto, that’s probably one of them. Obviously, Bell Labs and the transistor, that’s one of them. There are these really epic moments where a technology is pioneered and advanced well in advance of us really having a fully detailed understanding of all the ways that it’s going to affect us and do things. And I get to be a part of that. I get to be a part of that history. It’s not happening just at one company, it’s happening in the entire industry right now. And it’s, yeah, it’s daunting. It’s terrifying. But it’s obviously also a tremendous honor and privilege.

I want to acknowledge the moment we’re in too, with this rebrand for Facebook and now Meta. You all are no stranger to scrutiny. I think it’s probably safe to say you might be the most scrutinized company in the world right now?

Yeah.

It’s probably safe to say. There’s been these Facebook Papers the last few weeks. There’s already these conspiracy theories that you guys did this rebrand at this time to try to change the narrative. I know that’s not the case. I interviewed your boss, Mark [Zuckerberg], and he told me this started over six months ago, actually.

I mean, if we wanted to do that, we would have not done it this way. If that was the goal, then we would have taken a very different approach.

But acknowledging the moment we’re in, I mean, what do you say to the people who say this is a change to distance the brand tax that exists with the Facebook name now, for certain people and for lawmakers and the press, from everything else you all are doing?

Yeah, it’s not that. I mean, everything that we do is centered around consumers and consumer expectations. So we really were starting to hit these unusual spots. I’ll give you an example. So we had Oculus accounts. And part of the problem with Oculus accounts was that people really weren’t building up any kind of network to connect with. And what we do know is that when people have a network, they have more fun. For the same amount of time spent in the headset, they enjoy it more. That’s what they tell us.

And so we went to pretty elaborate extents to try to get, “Okay, let’s use the Facebook network in Oculus.” It’s an odd fit. It was an odd fit. And now Mark has announced that we’re going to change the way that we do accounts in Oculus.

So we’re trying to solve a problem and the only solution available to us isn’t a great solution. This kind of thing was happening all over the place. Facebook is a product. Having it try to also be an umbrella brand, which we tried, obviously, for the last several years, was really a struggle for us. It was really a struggle for consumers. I don’t think consumers really had a strong mental model of how that works.

If you ask consumers about Instagram and WhatsApp, and do you want to link these things or not link them, they understand what that means because those are products that they can have a sense of. And so I think we want to be able to have things like accounts that are at the Meta level,  but still give consumers a really strong understanding of how products relate to the data that they’re giving up, who they’re connecting to, what value they’re getting in exchange for all of that, make sure they feel good about it. So I think all these things are very consumer-friendly.

That’s a very practical reason to do it. The second part of it, for me, is — I know that it’s cheesy. No one wants to just run with the story we’re telling. But it’s the real damn story. It’s an exciting vision of what comes next. To some degree, we have hit the natural saturation point with mobile phones, with the mobile internet, with social networks. They kind of are what they’re going to be. There’s a lot of them, it’s very competitive, but they’re competitive on the margins. But we’re not seeing big steps forward and totally new things as much anymore.

And I think the metaverse feels like something that doesn’t exist today, and you can’t do it any other way. It’s in pockets, there’s little glimpses of it. And I think we’re excited about that. But the things that we’re describing are mostly just not possible without tremendous investment. And so I think for us, the corporate umbrella of Facebook served us so well for such a long time, because it was itself an unfulfilled vision. There’s still a lot of work to do there, obviously, but now we have a new unfulfilled vision that I think can power us for, let’s say, the next 15 or 20 years.

As you guys are doing all this and trying to lean into the future and trying to position the company that way, do you feel under siege at all, from the outside, in terms of the barrage of what is hitting you all? Is it distracting? Are you having to constantly tell people, “Look, we need to focus on what we’re trying to build.” Because people’s families are asking about these Facebook Papers, or what have you? How do you navigate that?

Yeah, I think it really depends on what your job is and where you are. Obviously, I think the comms team is pretty busy these days. I think for people who work in the fields that are under discussion, if you’re in research or in integrity, it’s hard because we don’t think these tell a particularly objective story. These weren’t random documents. These were specifically selected documents. They’re curated. There’s a selection bias that went into the documents.

I was selfishly disappointed there was nothing from Reality Labs in the document dump.

Oh yeah. You can imagine my disappointment. Not.

I think it depends on where you work. I think for those teams who are having their work maybe mischaracterized or misunderstood, I think that’s a challenge that they’re going through, the personal struggle. And obviously that’s part of it. But for a lot of people, yeah, they’re able to stay pretty heads-down I think. My team, we talk about it. We have a very frank discussion about, “Hey, here’s what’s out there. Here’s the reality as we see it. Here’s what you can do to follow up.”

And I think people are pretty satisfied with that. They’ve got the ability to talk to their friends and family and they can point them to the posts that Mark has made and say, “Hey, here’s the post, here’s the thing,” if you’re asking the questions. But it affects everyone differently. And a lot really does vary by the team and the role that that people play. So it’s not a uniform thing. We’re such a big company, nothing’s really going to effect us in a uniform way.

As part of this you guys are also getting rid of the Oculus name. I saw you announced that shortly after Connect, and you are replacing it with Meta. Portal, which is your video chat device for the home. It’s going to be called Meta Portal, Meta Horizon. Particularly with Oculus, which I think is probably the more established thing you guys have for VR, why do that?

It’s established and it’s beloved, both internally and externally. So this is kind of a heartbreaker. At some point, and again, the center is always on the consumers, and at some point, you want to have consumers have tremendous clarity about what is the company that’s providing this product. And then what are the services on that product and what can they use. And you just can’t have so many brands. We all get out there on the internet and make jokes about companies that have eight or 10 brand names in a row for everything they do. But then when it’s our brands, like, “Oh no, no. Don’t, that one, we love that one. That one’s a special one for us.”

So to some degree you have to walk the walk and say, “Hey, I love the Oculus brand.” And I’m sad that we are moving away from it. I’m going to miss it. The community is going to miss it. It’s funny. That was the number one reaction in my Twitter and Instagram mentions was just sadness. There was a real turning point for virtual reality at Oculus. But we want to make sure that we have a very good clarity with consumers. This is a Meta product, so that they understand what that is and build the equity there in their minds. So yeah, we had to move in a different direction.

So in the future, I’m going to have a Meta account, and I can choose to log into my Meta Quest with that?

We don’t have all the details, and I don’t mean we’re not sharing the details. I mean, we don’t have all the details yet on the direction we’re going to go. I think it was important for Mark to signal that as part of this change we are recognizing that it was a mistake to go bundle Facebook accounts with the Oculus Quest.

You got a lot of blowback when you did that.

Yeah. And again, it all came from a good place. Like I said, we had a vision. We knew something was going to be very pro-consumer on the other side of it. And it hasn’t been a factor for sales or engagement or things that indicate to us that people care about the product and are enjoying it and using it. But In the community that we love, our enthusiast community, that was a controversial change. And some parts of it have played out as we’d hoped they would. People are spending more time being more social. But some parts have played out in a different way. And so it’s been a little bit confusing and we just, we never want to have any confusion. My number one responsible innovation principle that we work with is “Never surprise people.” That’s number one.

And the Facebook thing surprised people. I mean, when you guys rolled it out. It did.

But I mean in the product, I mean in people’s expectations of a product that they’re using as consumers. And so, yeah, for me, that’s where we start and finish in this thing. So for the account stuff, we don’t have ... It’s not all settled how it’s going to be structured or how it’s going to be named or anything like that. Since this became clear to us, we’ve been working on this name change for six months. Since that moment, we started working internally on, “Okay, if we’re doing that, then the account system also can change finally.” We can actually accomplish our goals and not force the bundling. So we think it’s going to be really good for consumers, obviously more details as we have them. And so we’re well underway on the work there, but a lot of the consumer-facing work, what it’s going to be called and how people are going to access and relate to it, is to be determined.

This whole metaverse concept, I talked a lot with Mark about this. We can keep it pretty high-level. The idea is this immersive, embodied internet that is 3D. You could think of today like Roblox or Fortnite, where people are hanging out as avatars. And you guys are wanting to build this in AR and VR. I’m curious about how you take the concepts that we know today, a lot of them that you helped invent at Facebook and social media, and translate that to the metaverse. Is there going to be a News Feed in Horizon, which is this metaverse platform, software platform, you’re building? How are you thinking about the ways people engage with content in the metaverse?

I don’t think there will be a literal News Feed, except there might be your actual News Feed from Facebook. There’s no reason that 2D interfaces aren’t going to be an important part of an immersive metaverse in the same way that they’re an important part of how we navigate the physical world. But yeah, of course. There’s going to be so much to do. And in some ways, if you think about when you go to a city, there’s so much to do in a new city. How do you figure out what you want to do? There’s entire services, entire industries designed to help you navigate the amount that there is to do. 

There’s going to be way more to do in the metaverse, especially when you can instantaneously travel to any of the many cities that we kind of imagine ultimately populating the place. You’ll definitely need to have services that help you with what’s new, what’s hot, what’s trending, and what’s going on. What are other people doing? What are your friends doing? How can you plan things? Can you schedule things? So all those services are going to exist and we’re super excited about them. But I do think that it’s a little bit cart before the horse. Before I can figure out how I need to rank content for you, I need to have content for you. That’s just the sequencing that has to happen.

When you think about Facebook, how it allows one-to-many broadcasting, someone can instantly reach millions of people fueled by algorithms. Do you think that kind of value will translate to Horizon? For example, do I need to be the friend of someone or a friend of a friend before I can even interact with them in Horizon? Are you thinking about making it kind of more of a closed social graph, or is that still to be determined?

It’s still to be determined, but you’ve struck on something very interesting. There will be use for these asynchronous modalities that we’ve developed for the web and for the phone. And some will be 2D and some will be 3D and that’s going to be great. And there’s lots of things that you can do in the metaverse by yourself as experiences and that’s going to be great. But a lot of it is going to be quite new because I think the metaverse is largely a synchronous experience. You’re there with people in real time having an experience in real time. There’s something meaningful about it. That’s a trait that it possesses that is actually hard to get at in the internet that we have on mobile phones and on the web. You get touches of it. Look, here we are doing a podcast. That’s a cool synchronous thing. And people do video calls. They don’t love it. It’s not great, but we do that.

But for social, for fun, it’s not generally the way that we have these shared experiences today. So I do think that a lot of things are going to be different. A lot of the ideas that we’ve had to pioneer on the web won’t translate, and you have to completely rethink them. Some of them, it’s nice because you’ll be able to deploy metaphors from the real world, the physical world. That’s great. Some of them will be completely novel because there isn’t an analog in the physical world and that will require real invention. And then to your question about the friends thing, yeah, because it’s a synchronous environment, I think how we as a society want to think about integrity, privacy, is an open question. The stronger the integrity and privacy guarantees I want to make, the weaker the interoperability is going to be. And we know, that’s a tough tradeoff that we have to balance.

And likewise, if we make really strong privacy guarantees, that could trade off directly against integrity. So if there’s a private conversation and we’re not able to hear it, how do we manage the situation? One of the things that we do know we’re going to do, which I’m enthusiastic about, is give every individual who comes to the metaverse the tools to control their own experience. So absolutely, the idea of being like, hey, I’m only going to be visible [to], and only visible to me will be people who are within certain spheres. Or here’s a person who I don’t like, I mute them. They now don’t exist. And unlike on the internet where there’s kind of just an endless stream of accounts, that’s not how it is in a physical environment. There’s a real cost with switching accounts. And so I think there are some advantages that the synchronous environment is going to have in terms of an individual being able to feel in control of themselves and their experience and feel safe.

You talked a little bit about crypto and NFTs at Connect. It was the first time I’ve really heard Mark even say the word NFT publicly. I’m curious what you think of the promise of smart contracts, blockchain technology — I’m thinking of DAOs, decentralized autonomous organizations — can do in the metaverse, in an embodied internet where I’m an avatar with a bunch of people in an asynchronous environment?

When you think about interoperability, entitlements, as we would call it in the game industry, is an important concept. Like who owns this thing and what rights do they have to it? Can they make copies of it? Can they sell it? Can they take it with them into every single location? What locations can they take it into? And that’s not hard to do with the database if the entire metaverse is controlled by one company, which we don’t want it to be. And we don’t think it can be. It’s not necessarily hard to do with a database if we have a really good standardized schema and you can pick which database it’s in and it goes here and there and everyone kind of has access to that. But another way to do it, that could be really strong, is NFTs and crypto, where you’ve got the ledger.

Instead of having to store it in a database somewhere, which has its own downsides, you store it in the blockchain. And there’s an ability to say, yep, the system can verify that I’m the owner of this object and I have the right to make copies of it, or to sell copies, or whatever the thing is. So there’s an opportunity there. I think you know this, that space is very exciting and moving very quickly. The fundamental technologies are definitely one of the things that could be useful here. And I’d be very surprised if they weren’t one of the things undergirding at least part of it. I’m not sure that every part of the metaverse is going to be underpinned by crypto. But I think it’s important to support it. And especially given the amount of entrepreneurial energy in that space right now, this really is about creators.

The macro trend on the internet will carry us forward into the metaverse, I believe, which is just that creators, the creator economy that includes everything from developers and performers and people who are providing services. I think there’s going to be people who are doing interior decorating in the metaverse. They’re going to be decorating your home for you. There are going to be people who are stylists. You can get your avatar looking fresh. There’s going to be services. And so the creator economy is really the thing that will be the biggest thing to unlock. And I think there’s going to be concern from creators if the platform doesn’t provide them the degree of flexibility and control that they want.

I think this is getting at how maximalist and expansive this idea is, because I think when people think of the metaverse and Facebook, they think of a 3D version of Facebook and the Meta headset. What you’re talking about, especially using blockchain potentially, is taking it from one environment to the other. Say I want to move from Fortnite to Horizon. They’re both in VR. I want to take my avatar and all my virtual goods with me. Still, I’m very skeptical that you guys are going to figure that out with all the competitors in the industry. Maybe once this becomes just so realized that people have to do it because consumers are demanding it, then maybe. But building up to that, Apple, for example, they’re doing mixed reality. They’re going to have a headset. They’re going to have glasses. I think there’s zero chance they work with you all on interoperability in any kind of a virtual world. Maybe you disagree, but that’s the largest company in the world as an example. So how do you get people to actually buy into this?

There are a lot of levels here. I certainly think that you’re right. Just as the internet itself went through a lot of revisions and protocols that were designed but never adopted, and then these ones were adopted and different things happened. I expect the same to happen around the metaverse. And the majority of these questions are hard and they’re still ahead of us. Having said that, I don’t think we’re as far apart from most of the people in the industry as you might think. I think we all generally get a sense that if we can empower creators to have a richer economy, that creates a flywheel where more digital creation happens; that’s really good for consumers and that grows the economy, that grows the pie. We all benefit. One of the obvious things that you can do to increase the value that you’re giving to your developers, your creators — I’ve heard Roblox, I’ve heard Epic, I’ve heard a lot of people talk about this — is getting them a larger audience. That’s an easy one. It costs you nothing. They’re developing the same thing and now the audience is larger. So I think the watch word for the metaverse is continuity, being able to have a continuity of experiences across both experiences and platforms built by different companies. 

And so, yeah, there are areas where this is actually pretty workable. Let’s take avatars. Being able to implement somebody else’s avatar or having to implement your avatar for someone else’s system is actually pretty workable. It’s not an impossible challenge. Does that mean every avatar is going to be useful everywhere? No, of course not. But there are also clothes that I can’t wear in every place that I go in the physical world; that wouldn’t be appropriate.

I’d love to know what this is.

You know, I wouldn’t wear an Easter bunny costume to church. That seems like a mixed message. And so it’s not unheard of that we would have these cases where’s like, yeah, I can’t take this thing over there because of whatever rules that apply over there. So it’s not totally unheard of even in the physical world. So anyway, I bring this up to say, I agree it’s going to be hard. You’re absolutely right. If there’s a thing to be skeptical of, you nailed it. That’s the hardest part for sure. However, at least at a conversational level, whether I’m talking to people at Microsoft, talking to people at Google, different people, there is a vision that we share, I think, that is coming into focus for the industry. And if we can find really strong standards in a way that allows people to recoup their investments, because this is expensive work, as Mark said, then I think there’s a chance. There’s a path.

There’s a second path, which is the one that often works, which is you get enough consumers in one area. And then you’re able to attract more and more partners into the area to interoperate on that platform because they want to go where the marketplace is. And that’s another path that is possible.

I do want to touch on hardware because you teased a little bit more about two big hardware things. You’ve got Cambria, which is this high-end version of VR. It’s really mixed reality with full-color passthrough that’s coming next year. You’re going to share more about it next year. And then you’ve got Project Aria and Nazare, which are the AR glasses that are farther out. But I guess maybe we could start with Cambria. Is the idea that eventually all this is going to merge together, and so we had VR first, now we’re going to get passthrough experiences where I’m wearing a headset with video feeds looking out, kind of mixing graphics with the real world? And then eventually we get to AR, which is just in my glasses. Is it a continuity like that?

Yeah, from a sequencing standpoint you’re right. And you’re certainly right that there’s a nontrivial overlap in the Venn diagram of AR and VR, and we call it mixed reality. But I will point out the use cases are pretty different. If you’re putting on a big headset with a larger field of view that’s more occluded, that is a little bit more immersive, that’s the kind of experience that you might have at your house, at your desk, at your office. It’s not one that you’re going to have out in the street. It’s not one that you’re going to have on the train, walking around. For those experiences, you’re probably going to prefer AR. And the things that you’ll want to do when you’re mobile, moving around throughout your day, turn out to be pretty different than things that you’re doing when you’re sitting in one place.

So I think there is a permanent, lasting place for VR. I think VR in the future will always be augmented with mixed reality. It’s just safer, better for a bunch of things. It’s a good thing. It’s going to be a while before we get there just because right now it’s expensive. You’re adding cameras, adding the cameras themselves is the most expensive thing. But now you have to process all the image data and so that adds to the thermals, that adds to the processing envelope. And so I think, yeah, I think in the future there’s a lasting, permanent place for VR in our lives, in our workflows and the things that we do for fun. And then there’s also AR, which I think creates its own new category. But then of course you’re right, there will be areas that the two overlap at a software level.

There’s Nazare, which is the code name for these AR glasses that are pretty far out. You’ve got Project Aria right now which is doing egocentric data capture with no displays in them. You’re expanding that program now. You’ve been doing it for the last year or so where people are just walking around ID’d as being Facebook contractors and they’re just capturing what they see with sensors on their glasses. And this is to help you train how AR glasses will work in the real world, because it’s an entirely new point of view for compute. Could you talk a little bit about how that’s going? 

I think there’s two pieces here. One of which is the egocentric data that we’re capturing with Aria and using the training, yeah, there’s a lot of security in addition. We’re blurring faces and blurring license plates. The data’s not user-accessible to the person recording it, so on and so forth, just for your listeners who are worried. We’ve got a whole open program on that. The details are all available.

And that’s really critical because so much of what you want to do in augmented reality is contextual. You want it to have a sense of what you’re up to, what you’re doing. So when somebody walks up to me and they’re having a conversation, I need to  get all the UI out of my face so that I can see their face and they can see my eyes, and that’s really good. But maybe there is something that the system could know is so important that it needs to interrupt me for.

And so the more context that this thing has in understanding the physical world around me as well as the digital artifacts it’s managing, the better it’s going to serve me. Because again, the goal of this is to be both more present and more connected. So it doesn’t really buy us much, in the mission of Facebook at least, if you put these things in your glasses and now you’re just constantly on your phone, but it’s your glasses. That’s not exciting. That’s not the goal. The goal is to have it so that the information that’s relevant to you is there and accessible when you need it. And then it’s not there when it would otherwise be in the way or distracting you from a moment that you’re experiencing in your actual life.

That’s a high bar. We’re talking about a completely novel interaction design between people and machines that we haven’t had before. So I think for me, what we’re going to do is really focus on the AI research that comes out of the Aria dataset. And it’s a huge partnership with Facebook’s absolutely world-class AI team.

Meta.

To be fair, I think right now it is still the Facebook AI team, but you’re right. We probably got to chase that one down, all the way down. I gotta say, we changed the sign in front of the building today. We did a pretty good job of getting this thing out in one day without it leaking. I’m pretty pleased about that. There is going to be a six-month adjustment period here as we work through all the pieces.

And so anyway, for me the interaction paradigm is so novel it’s going to need a level of intelligent and contextual understanding, and you want to run as much as you can locally on the device. Not only is that the privacy-friendly way to do it, it’s also the thing that is probably required because you’d be otherwise sending so much data back, and data is expensive to transfer. Wireless radios are expensive to operate. We’d like to not operate them. We’d like to do as much of this locally as possible.

And so getting these models built that we can then miniaturize into silicon, possibly custom silicon, and then run locally on the device so that you can have this contextual awareness, this user interface that reacts really intuitively to what you’re trying to accomplish. That’s the North Star vision that we have and I think it’s exciting and it’s as hard a thing as anything else in here. We talk about these hard technical problems, but the user interaction design challenges are as hard as anything. There’s no keyboard, there’s no mouse, there’s no touchscreen. Direct manipulation is out. So what are we doing to try to replace it? What are we doing to make it so it’s not as required?

We don’t have time to talk about the CTRL-Labs stuff [neural interfaces] because you’re right, there is so much that you are working on. Lastly, do you still think AR glasses are three years or so out? 

We definitely hope to be playing with prototypes in the next couple of years. And this is one of the things that confounds the press and nobody believes me, it’s really true. You don’t decide, here’s the thing, I’m going to ship it at this point. You decide, here’s the thing, I’m going to build it and while I’m playing with it I will decide whether that’s a shippable thing or not a shippable thing.

It’s so funny. I posted these pictures of me and Mark in various prototypes, and those are research prototypes. Those weren’t even production prototypes. One was an industrial design prototype. It had no mechanics, no electronics in it. One was just for testing the boundaries of resolution. They’re prototypes. And people were like, “What is the product?” And it’s like no, you don’t understand. We just have headsets laying about the place covered in all kinds of gear to try out things. And then they converge relatively late, surprisingly late, I think, to the layperson. Whether we decide okay, that is a product that we have. We’re definitely going to be playing with some really good ... We have early, early, early prototypes right now of course, but we’re going to be playing with more robust prototypes in the next year, hopefully. Maybe two years.

All right, Boz. I’m going to let you get back to your giant mound of prototypes. Maybe you could tweet some more teasers.

I’ll do what I can, absolutely. Thanks, Alex.