PATRICK COLLISON: All right, good afternoon, folks. I hope you’ve enjoyed the sessions between now and when we last saw you this morning. For this afternoon’s keynote, or fireside chat, I suppose, I’m about to introduce somebody who needs little introduction. Although, a fun fact that you may not know about Jensen Huang is that he’s been a CEO of NVIDIA for 31 years this month, making him the longest-serving CEO in the technology industry.

So, John and I have only been doing it for a mere 14 years, so you know even if we double that, we’ll still be second to him. And, Jensen, we’ll talk about this on stage, attended the Oneida Baptist Institute in Kentucky. We’ll definitely be asking about it. Oregon State. Worked as a waiter at Denny’s, then… Denny’s close to here, actually. LSI Logic, and then AMD, which is of course now run by his first cousin once removed. We’ll definitely be asking about that.

Before he founded NVIDIA in 1993—and NVIDIA’s market cap was $8 billion when Stripe launched in 2011, and it is now, of course, more than 200 times that. So, he’s been busy since. Please, welcome to the stage, JENSEN HUANG.

JENSEN HUANG: Hey, everybody.

PATRICK COLLISON: So you watched the keynote earlier?

JENSEN HUANG: I did. I’ve never seen a duet before. You were so synchronized. It seemed like the two of you knew each other. It’s incredible.

PATRICK COLLISON: Some acquaintance… But, okay, you’ve been doing keynotes a long time. You are the keynotes GOAT, so…

JENSEN HUANG: Stop it.

PATRICK COLLISON: So give us your... we don’t have even a signature outfit yet. We’re just amateurs here. So give us...

JENSEN HUANG: It’s because you’re still young.

PATRICK COLLISON: Give us your keynote performance review. What’d you think?

JENSEN HUANG: I thought it was A+. I thought it was A+. I thought it was... really. You explained perfectly the purpose of the company, what inspires you guys, what keeps you guys up, what makes you work so hard, the ecosystem that you serve, the incredible platform you’ve built, the amazing contribution you make to the world’s economy. It’s incredible. I thought it was great. And there was a whole bunch of technology stuff, feature stuff, money stuff. I didn’t understand any of that, but... But something about a CYK or something. What was that?

PATRICK COLLISON: KYC.

JENSEN HUANG: KYC. Yeah. I thought it was...

PATRICK COLLISON: It’s a big deal in our world.

JENSEN HUANG: Is that right? Kentucky Fried Chicken?

PATRICK COLLISON: We take care of KYC so that you can associate us with Kentucky Fried Chicken.

JENSEN HUANG: Okay, got it.

PATRICK COLLISON: Did you... Software-defined financial services, this idea, did that...

JENSEN HUANG: Incredible.

PATRICK COLLISON: Does that make sense to you?

JENSEN HUANG: Well, first of all, I think it’s a giant idea.

PATRICK COLLISON: Do you know where it came from?

JENSEN HUANG: You’re going to tell me.

PATRICK COLLISON: So Jensen and I were catching up, maybe...

JENSEN HUANG: But the part that I loved was how you realized, in the very beginning, that financial payments was about code, not finance. I thought that was incredible. And you explained that the first time we met.

PATRICK COLLISON: So Jensen and I were catching up 18 months ago or so and I guess it was a couple years since we’d last spoken. So he was kind of asking for the update on Stripe. And I was explaining, and you said “Oh, so it’s like software-defined networking but for money.” That was still ricocheting around in my mind. So that’s where we got to this idea for software-defined financial services. So I hope we don’t have to pay a licensing fee for that or something.

JENSEN HUANG: I got zero equity for that good idea.

PATRICK COLLISON: All right, you guys are doing okay. I was thinking about this. Tesla’s earnings were of course yesterday and Elon announced that I think Tesla is going to have 85,000 H100s by the end of this year. I was just reflecting on... it’s quite a success to sort of build a business where CEOs kind of compete with each other to announce who has spent more buying your product. So, I think you’ve done something quite impressive. But anyway, I actually want to start out talking a bit about...

JENSEN HUANG: All of my CEO friends, they all have the most.

PATRICK COLLISON: I want to start out talking a little bit about a remark you made at a Stanford event recently. I was thinking of GSB, I think. You said, “I wish upon you ample doses of pain and suffering.” Elaborate.

JENSEN HUANG: Well, let’s see. There is a misunderstanding. There’s a phrase that said, “You should choose your career based on your passion.” And usually, people connect passion with happiness. I think there is something missing in that. Nothing there is wrong, but there’s something missing. And the reason for that is because, if you want to do great things and I know this to be true about you creating Stripe. By the way, this is one of the world’s finest CEOs. Young as he may be. You guys know I’ve met a lot of CEOs. I’ve heard about a lot of companies and this is genuinely one of the world’s great visionary companies. So anyways, I just wanted to say that. And it’s the reason why I’m here, I just love what...

PATRICK COLLISON: No more compliments allowed. It makes us terribly uncomfortable.

JENSEN HUANG: I know. I could tell. I could see him. He’s starting to sweat. And so, the thing is, when you want to build something great it’s not easy to do. And when you’re doing something that’s not easy to do, you’re not always enjoying it. I don’t love every day of my job. I don’t think every day brings me joy nor does joy have to be the definition of a good day. And every day, I’m not happy. Every year I’m not happy about the company, but I love the company every single second.

So I think that what people misunderstand is somehow the best jobs are the ones that bring you happiness all the time. I don’t think that that’s right. You have to suffer. You have to struggle. You have to endeavor. You have to do those hard things and work through it in order to really appreciate what you’ve done.

There are no such things that are great that were easy to do. So by definition, I would say therefore I wish upon you greatness, which by my way of saying it, I wish upon you plenty of pain and suffering, and so.

PATRICK COLLISON: Anything in your upbringing that taught you that idea? Or is it just somehow innate to your makeup?

JENSEN HUANG: I didn’t realize I had to lay down for this but... I’m about to tell you things I’ve never told anyone, not even my family. I was an immigrant. And when I came in 1973, I was 9. My older brother was almost 11. This was a foreign country and there was nothing easy about that. We also grew up in a... really, really terrific parents, but we weren’t wealthy. And so, they worked hard. They work hard today. So they passed along a lot of life lessons by working hard. Now, I had all kinds of jobs. We went to a school that included a lot of chores.

PATRICK COLLISON: It was in Kentucky.

JENSEN HUANG: Yeah. Kentucky, Oneida Baptist Institute. And I... I don’t think it’s the same as MIT that “I” is not the same. It’s the same word, but it’s different. It’s a different type of institute. But my institute required you to go to school, and it was a dormitory, and so there were a lot of chores. I was the youngest kid in school and so all of the other kids got the hard work. They had to work in the tobacco farm. I got the easy job. I was 9 years old. So after they left, I had to clean all the bathrooms.

I never felt that I got the easy job because what they left behind was... you can’t unsee that kind of stuff. But that was my job and so I did it delightfully. Then I had plenty of other jobs, and Denny’s was one of them. I started out as a dishwasher and became a busboy and became a waiter. And I loved every one of them. I loved every one of them.

Somehow, I’ve always found... I want to say joy, but that’s not quite right. I just, everything that I was doing, I wanted to do the best I could. And maybe that was kind of ingrained from the very beginning but I was definitely the best bathroom cleaner the world’s ever seen, I’m sure of it, yeah.

PATRICK COLLISON: So if we fast-forward just a little bit to the NVIDIA of today. How large is your leadership team?

JENSEN HUANG: NVIDIA’s leadership team is 60 people.

PATRICK COLLISON: And they all report to you?

JENSEN HUANG: Yeah, they all report to me.

PATRICK COLLISON: Sixty direct reports.

JENSEN HUANG: Sixty direct reports, yup.

PATRICK COLLISON: Which is not conventionally considered a best practice… (Audience laughs.) I agree that the best practice kind of...

(Audience laughs.)

JENSEN HUANG: I’m certain that’s the best practice. It’s not conventional, but I am certain it’s the best practice. (Audience applauds.) By the end of this, I’m going to convince all of you to have 60 people on your direct reports.

PATRICK COLLISON: The floor is yours.

JENSEN HUANG: First of all, the reason is because the layer of hierarchy in your company really matters. Information really matters. I believe that your contribution to the work should not be based on the privileged access to information. I don’t do one-on-ones and my staff is quite large. Almost everything that I say, I say to everybody all at the same time.

The reason for that is because I don’t really believe there’s any information that I operate on that somehow only one or two people should hear about. “These are the challenges of the company,” or “This is the problem I’m trying to solve” or “This is the direction we’re trying to go into. These are the new endeavors.” “This isn’t working. That’s working well.” And so all of this type of information, everybody should be able to hear.

I love that everybody’s working off of the same song sheet. I love that there is no privileged access to information. I love that we’re able to all contribute to solving a problem. And when you have 60 people in a room and oftentimes, my staff meetings are once every other week, it’s all based on issues, whatever issues we have. Everybody’s there working on it at the same time. Everybody heard the reasoning of the problem. Everybody heard the reasoning of the solution. Everybody heard everything.

And so that empowers people. I believe that when you give everybody equal access to information, it empowers people. And so, that’s number one, empowering. Number two if the CEO’s direct staff is 60 people, the number of layers you’ve removed in a company is probably something like 7. Depending on how it is.

PATRICK COLLISON: Is it 60 at every layer? As in, if I’m one of the fortunate 60, do I also have 60 direct reports?

JENSEN HUANG: No.

PATRICK COLLISON: Okay.

JENSEN HUANG: I don’t think that that’s scalable downward. And the reason for that is because you need more and more supervision depending on certain levels. And at the E-staff level, if you’re so unfortunate to be serving on NVIDIA’s E-staff, it’s very unlikely you need a lot of managerial.

PATRICK COLLISON: So I rarely find myself having to... stand up for conventional wisdom. But if I were to steel man the other side, I’d say, “Well, one-on-ones are where you provide coaching, where you maybe talk through goals together, personal goals, career advancement, what have you. Where maybe you give feedback on something that you see somebody systematically not doing so well and so forth.” And there is all these things that one is, again conventionally supposed to do in the one-on-one. Do you not do those things, or do you do them in a different way?

JENSEN HUANG: Really good question. I do it right there.

PATRICK COLLISON: Right there in...

JENSEN HUANG: I give you feedback right there in front of everybody. In fact, this is really a big deal. First of all, feedback is learning. Feedback is learning. For what reason are you the only person who should learn this? Now, you created the conditions because of some mistake that you made or silliness that you brought upon yourself. We should all learn from that opportunity. So you created the conditions, but we should all learn from it. Does it make sense?

And so, for me to explain to you why that doesn’t make sense or how I differ from it, half the time, I’m not right. But, for me to reason through it in front of everybody helps everybody learn how to reason through it. So the problem I have with one-on-ones and taking feedback aside is you deprive a whole bunch of people that same learning. Learning from mistakes, other people’s mistakes, is the best way to learn. Why learn from your own mistakes? You know, why learn from your own embarrassment? You got to learn from other people’s embarrassment. That’s why we have case studies, and isn’t that right? We’re trying to read from other people’s disasters, other people’s tragedies. Nothing makes us happier than that.

PATRICK COLLISON: Have you succeeded in getting other leaders at NVIDIA to adopt this practice? Or is that difficult?

JENSEN HUANG: I give people the opportunity to decide for themselves but I really discourage one-on-ones. I really discourage one-on-ones. Nothing is worse than the idea that somebody says “Oh, Jensen wants us to do this.” Why does that have to be said to anybody? Everybody should know. Or, “E-staff said that.” Nothing drives me nuttier than that.

PATRICK COLLISON: You once told me that you really didn’t like firing people and very seldom did it. Can you elaborate on that?

JENSEN HUANG: Well, I’d rather improve you than give up on you. When you fire somebody, you’re kind of saying... Well, a lot of people say “Well, it wasn’t your fault,” or “I made the wrong choice.” There are very few jobs. Look, I used to clean bathrooms, and now I’m the CEO of a company. I think you could learn it. I’m pretty certain you can learn this.

There are a lot of things in life that I believe you can learn and you just have to be given the opportunity to learn it. I had the benefit of watching a lot of smart people do a lot of things. I’m surrounded by 60 people that are doing smart things all the time. They probably don’t realize it but I’m learning constantly from every single one of them. So I don’t like giving up on people, because I think they could improve. So there’s... It’s kind of tongue in cheek, but people know that I’d rather torture them into greatness.

PATRICK COLLISON: That was the phrase that I was hoping to uncover. Yeah, I remember you mentioned that.

JENSEN HUANG: Yeah. So I’d rather torture you into greatness because I believe in you. And I think coaches that really believe in their team torture them into greatness. And oftentimes they’re so close. Don’t give up. They’re so close. Greatness, it comes all of a sudden. One day it’s like, “I got it.” Do you know what I’m saying? That feeling that you didn’t get it yesterday and all of a sudden one day something clicked, and “Oh, I got it.” Could you imagine if you gave up just that moment right before you got it? So I don’t want you to give up on that. So let’s just keep torturing you.

PATRICK COLLISON: How’s your work-life balance?

JENSEN HUANG: Well, it depends on who you ask. I think my work-life balance is really great. It’s really great. I work as much as I can. I feel like he’s judging me. You know, I’m older than you. I have more wisdom than you. So what I...

PATRICK COLLISON: These are all the highlights from our conversations that I think more people should get to hear, so.

JENSEN HUANG: Well, I work from the moment I wake up to the moment I go to bed. And I work seven days a week. When I’m not working, I’m thinking about working. And when I’m working, I’m working, and so. I sit through movies, but I don’t remember them because I’m thinking about work. You know? And so that’s... but my work is not as, you know... It’s not “working” as in... There’s this problem, and you’re trying to solve this problem. You’re thinking about what the company can be and are there things that we could do even better. Or sometimes, it’s just trying to solve a problem. But sometimes you’re imagining the future and boy, if we did this and that. And it’s working. You’re fantasizing, you’re dreaming. I mean, that’s incredible.

PATRICK COLLISON: Well, so yeah, to concretize this a little bit and then, we will get to talk about AI, which I hear is a thing these days, but...

JENSEN HUANG: It’s a thing.

PATRICK COLLISON: Yeah. Officially a thing, TM. But, to concretize this a bit, what does a day in Jensen’s life look like? Like when do you wake up?

JENSEN HUANG: Well, I used to wake up at 5. These days I wake up at 6 because of my dogs. And the reason why, 6 is somehow we decided that 6 o’clock is when they should wake up. And I don’t know what it is. I don’t mind waking anybody up, but I feel guilty when I wake the puppies up. It actually burdens me. So I don’t want to move, it might... they pick up on any vibration in the house and it wakes them up. So we stay in bed and I just read in bed until 6 o’clock and it’s time.

PATRICK COLLISON: But you’re thinking about GPUs?

JENSEN HUANG: Oh, yes, yeah, yeah, sure. I’m obsessed about GPUs. I mean, what can you do? I’m constantly... no, I’m just...

PATRICK COLLISON: Then the day is all, I guess, group meetings because it can’t be one-on-one meetings.

JENSEN HUANG: Yeah. I get my work done before I go to work. And then when I get to work...

PATRICK COLLISON: How many meetings in a typical day?

JENSEN HUANG: Pretty much all day long. So I select the meetings that are really important to me. I try not to have regular meetings, regular operational meetings, because I’ve got amazing people in the company who are doing regular operational meetings. So we’re pinch hitters. CEOs are pinch hitters. We should be working on the things that nobody else can or nobody else is.

PATRICK COLLISON: So you’re jumping into projects that are stuck or offtrack.

JENSEN HUANG: That’s right.

PATRICK COLLISON: Or new ideas.

JENSEN HUANG: Wherever we can move the needle. No reporting. No reporting meetings. I hate reporting meetings. They don’t want to report to me, and just problem meetings. So problem meetings or idea meetings or brainstorming meetings or creation meetings or whatever it is. Those are the meetings I go to. Usually I call them. I try really hard not to have Outlook manage my life. We purposefully decide what kind of things that we want to do, we want to work on. So I try to live a life of purpose and I manage my time accordingly.

PATRICK COLLISON: You used a phrase once “Zero-billion-dollar markets,” that zero-billion-dollar markets are your favorite markets.

JENSEN HUANG: Yeah.

PATRICK COLLISON: What do you mean?

JENSEN HUANG: If you take a step back, our purpose, almost all of our purposes should be to go and do something that has never been done before that is insanely hard to do—that if you achieve it, it could make a real contribution. I know your company does that. I try to do that. If that’s the case, it hasn’t been done before, it’s incredibly hard to do. That market is probably zero billion dollars in size because it has never been done before.

I’d rather be a market maker, market creator than a market taker. You know, to create something new that never existed before versus thinking about share. I don’t love thinking about share. I don’t like the concept of share. The reason for that is because if you think about it in the big picture, Stripe existed out of thin air. You vaporized. You created something out of vapor. It wasn’t as if there was another... something else.

So I’d like to think that we can come up with something that is zero billion dollars. A zero-billion-dollar market is a good way to cause the company to think about how to go create something for the first time.

PATRICK COLLISON: So our mission is to grow the GDP of the internet. I mean, the GDP of the internet—a clause in that usually gets most of the attention. But I think the most important part is just the verb grow. Yeah. Because, to your point we shouldn’t be thinking about, well... which are the transactions that are already happening or which are the businesses that already exist. We should be thinking about which are the transactions that don’t exist.

JENSEN HUANG: That’s right.

PATRICK COLLISON: Which are the businesses that don’t exist.

JENSEN HUANG: Exactly.

PATRICK COLLISON: The GDP of the world is around $100 trillion but it doesn’t have to be $100 trillion. It could be $200 trillion or $1,000 trillion.

JENSEN HUANG: That’s exactly right. That’s exactly right. Most of the value we’re going to create over the next several decades are likely not limited by physical things. So this is a pretty extraordinary time.

PATRICK COLLISON: So with this concept of zero-billion-dollar markets if I’m, again, at NVIDIA am I coming to you with some proposal for some project and maybe there’s several billion dollars of CapEx involved or, you know, it’s a many-year pursuit or something and there are no customers for it today. There’s no demand that I can demonstrate for it. And you guys are just making a gut call to say that “Yes, nobody’s doing this today. We think they could. We think they should. And therefore, we’re going to pursue it.”

JENSEN HUANG: Really close. Yeah, it’s kind of like that. It’s a gut call in the sense that your intuition says something as a starting thesis but then you have to reason through it. And the reasoning of it is much, much more important to me than a spreadsheet. I hate spreadsheets because you can make spreadsheets do whatever you want. You can make any chart you want out of a spreadsheet. You just got to type in some numbers.

So I don’t love spreadsheets for that reason. I love words for that reason. Words are reasoning. Tell me, how did you reason through this? What’s our intuition? Why do we believe that matters? Why do we think it’s hard? I like hard things, because it takes a long time to do. And if it takes a long time to do, a lot of people who are less committed probably won’t do it.

If it’s really, really hard to do, it takes a long time to do, it takes us a really resilient and a really dedicated, really committed person to go after it. And if it also takes a long time to do you can kind of flounder around for a couple years nobody notices. So I could be incompetent for several years, and everybody goes, well, who saw it?

PATRICK COLLISON: And where did CUDA come from?

JENSEN HUANG: CUDA came originally from two ideas. One is called... I hate to get technical, but we created—we pioneered this idea called accelerated computing. Accelerated computing is like an IO device, something that you sit on PCI Express, if anybody’s in the computer business, an IO device that allows the application to interact with that IO device in such a way as to accelerate parts of the application.

And UDA was an invention in 1993, and this really profound invention allows the software programmer to directly program an IO device, write an application directly to the IO device because the IO device is virtualized and it’s... architecturally compatible across multiple generations. Anyways, we invented this idea called accelerated computing and that was, we called it Unified Driver Architecture for whatever reason.

And then, several years later we thought we could make our GPUs more programmable to high-level programming languages and we invented this idea called CG. C for graphics. C for graphics processors. That opened up some really exciting opportunities. And we thought, you know what? This is going to work, but CG, the programming model, wasn’t exactly right. And so we extended, we invented CUDA, which is compute with... So anyways, that’s how. It’s a horrible story, frankly. Anyways, we invented this idea called accelerated computing. We pioneered this approach.

PATRICK COLLISON: I guess the real question is, was it a smash hit overnight?

JENSEN HUANG: No, it was a... It was an incredible disaster overnight. And it kind of went like this.

PATRICK COLLISON: So this is one of your zero-billion-dollar markets you went after.

JENSEN HUANG: Yeah.

PATRICK COLLISON: And it was a disaster.

JENSEN HUANG: Yeah. Because it was a zero billion dollar we went after, but it cost so much to go after that zero-billion-dollar market it actually crushed the $1 billion market we were enjoying. So, and the reason for that is because CUDA added a ton of cost into our chips. But there were no applications. And there are no applications. Customers don’t value the product, and they won’t pay you a premium for it. And if people aren’t willing to pay you for it but your cost went up, then your gross margins get crushed. And we got... Our market cap was low, and it went down to really low. It was like, I think our market cap went down to like a billion dollars or something like that. I wish I bought it, but anyways…

PATRICK COLLISON: Okay. So therefore, you immediately canceled CUDA and went back to the old strategy.

JENSEN HUANG: No, no, I believed in CUDA because you reasoned about it. You reasoned about it. Look, we really believe that accelerated computing was going to be able to solve problems that normal computers couldn’t. And if we wanted to extend the architecture to be much more general purpose, we had to make that sacrifice. So I deeply believed in the mission of our company. I deeply believed in its opportunities.

PATRICK COLLISON: And so were analysts...

JENSEN HUANG: And I deeply, deeply believed that people were wrong. They just didn’t appreciate what we built. I deeply believed it.

PATRICK COLLISON: And so, weren’t analysts and the board and employee like, “You’ve torpedoed this existing revenue stream. You’ve this hyped thing that... you’re selling a lofty dream around that nobody seems to actually want. The business is really suffering.” Talk us through that. You believed.

JENSEN HUANG: You just go something like this. “Oh gosh, they’re so dumb.” Something like that. You know, denial. No, I’m just kidding. No, you go back to what you believe. And if you believe something...

PATRICK COLLISON: Did the board put pressure on you during this?

JENSEN HUANG: I’d start every conversation with what I deeply believed. And they believed it because they saw me deeply believe it. And I reasoned about it. It wasn’t like it was a spreadsheet and therefore, you’ve got to believe the spreadsheet. They had to believe the reasoning, the words.

PATRICK COLLISON: How long did it take it to start working?

JENSEN HUANG: Probably 10 years, yeah. Yeah. It wasn’t that long. Yeah. Ten years. It comes and goes. Ten years.

PATRICK COLLISON: Less than a third of your tenure.

JENSEN HUANG: Yeah, it comes and goes. It was, I barely remembered it. The suffering, I barely remembered it.

PATRICK COLLISON: Could NVIDIA be as successful in AI without CUDA?

JENSEN HUANG: No. Impossible. It is potentially one of the most important inventions in modern computing. We invented this idea called accelerated computing. And the idea is so simple, but deeply profound. It says the vast majority... a small percentage of the code of programs occupies, consumes 99.999% of the runtime. This is true for a lot of very important applications.

That small little kernel or, you know, some, several kernels... can be accelerated. And they tend... It’s not all just parallel processing. It’s not as simple as that. But the idea is that we can take that kernel, that piece of software, that part of the software and accelerate the living daylights out of it.

And today, when Moore’s Law has run its course and CPU scaling is basically stopped. If we don’t accelerate every software, you’re going to see extraordinary computation inflation. Because the amount of computation the world does is doubling every year still, and yet, if CPUs and general-purpose computers are not increasing in performance because it’s stopped, then what’s your alternative? Or your cost of computing is going to keep going up exponentially. So the time has come for us to do that.

PATRICK COLLISON: So everyone here runs a business, and...

JENSEN HUANG: Accelerate everything.

PATRICK COLLISON: And you heard it here first. And probably everyone has... some version of CUDA or a thing that they think really makes sense for the sector or makes sense for their technology or what have you but where the market doesn’t see it yet. Do you think it’s possible to extract any kind of generalizable principles around when you should really doggedly trust that vision and when perhaps it’s worth reconsidering in a fashion that, yeah, we could extrapolate from, in the case of CUDA and other CUDAs that have existed over the course of NVIDIA’s history?

JENSEN HUANG: Yeah, the question is determination and commitment versus stubbornness. And that line is fuzzy. Look, I gut-checked against my core beliefs every day. I still do. And you gut-check against it. The first principles by which you reasoned about your strategies, those first principles are easy to remember. It’s not a long list.

Now the question is, did those principles... did they change in some fundamental way? Are external conditions such that they no longer matter as much as before? Did somebody else solve the problem and therefore, that problem has now disappeared? Is it, there will never be any need? You gut-check it constantly to the extent that that’s number one, gut-check. You have to, first of all, you really have to be careful to distill down the first principle instead of “I want something.” That’s stubbornness. You can’t reason about it. I just want it. We’re not 5-year-olds, right?

So you got to reason about it, number one. Number two, you have to be clever. The fact of the matter is there are a lot of new companies being created here. It’s amazing how many great companies are in the audience and young companies in the audience. You have to be clever. So we found ways to monetize even in a small way, CUDA.

And so, we found app, we looked everywhere for applications. We found an application with CT reconstruction. We found an application with seismic processing. We found another application with molecular dynamics. And so we’re constantly looking for applications. They didn’t make it a home run but it sustained us just enough, just enough and bought us time for it to really happen.

PATRICK COLLISON: Okay, so let’s talk about AI. Maybe just going to do some math to ground things here. Let’s just say that the total compute capacity of all GPUs in the world today is X. What multiple of X will we be at in five years?

JENSEN HUANG: First of all, you know that I’m going to regret saying this. And this is... I’m a public company, you crazy person. See, this is... how nice is it to be private?

PATRICK COLLISON: Safe to say considerably more?

JENSEN HUANG: Well, let’s reason about it, shall we? Okay, so let’s reason about it. Let’s reason our way through, okay? So first of all, it goes like this. The world has installed about a trillion dollars worth of data centers. Those trillion dollars worth of data centers used as general-purpose computing. General-purpose computing has run its course. We cannot continue to process that way. And so, the world is going to accelerate everything data processing, you name it, okay? So we’re going to accelerate everything.

When we accelerate everything, every single data center, every single computer will be an accelerated server. Well, there’s about a trillion dollars worth of computers if we don’t grow at all over the next, call it four years that we have to go replace. Four years, six years, pick your number of years. But, if the computer industry continues to grow at some 20% or so, we’ll probably have to replace over the course of next... pick your number of years, about $2 trillion worth of computers with accelerated computing. So, just make that GPUs, okay? That’s number one.

And this is the second part. This is the reason why all of you, Stripe you’re onto something just absolutely monumental. This idea called, and... you’ve heard me say an industrial revolution. Let me tell you why. We are producing something for the very first time that has never been produced before. And we’re producing it in extremely high volume. And the production of this thing requires a new instrument that never existed before. It’s a GPU.

The thing that we’re producing for the very first time, for the mathematicians and all the computer scientists in the room, for all of you know that we’re producing tokens. We’re producing floating point numbers at high volume for the first time in history. The floating-point numbers have value. The reason why they have value is because it’s intelligence. It’s artificial intelligence. You can take these floating-point numbers, you reformulate it in such a way that it turns into English, French, proteins, chemicals, graphics, images, videos, robotic articulation, steering wheel articulation. We’re producing tokens at extraordinary scale.

Now, we’ve discovered a way through all of the work that we do with artificial intelligence to produce tokens of almost any kind. So now, the world is going to produce an enormous amount of tokens. Now these tokens are going to be produced in new types of data centers. We call them AI factories.

Back in the last industrial revolution, water comes into a machine, you light the water on fire. Turn it into steam and then it turns into electrons. Atoms come in, electrons go out. In this new industrial revolution, electrons come in and floating-point numbers come out. And just like the last industrial revolution, nobody understood why this electricity is so valuable and is now sold, marketed kilowatt hours per dollar. And so, now we have million tokens per dollar.

That same logic is as incomprehensible to a lot of people as the last industrial revolution, but it’s going to be completely normal in the next 10 years. Well, these tokens are going to create new products, new services, enhanced productivity on whole slew of industries, a hundred trillion dollars worth of industries on top of us. So this industry is going to be gigantic. In order to monetize that, transact that you’re going to need Stripe.

I got to tell you: this is one of my favorite companies. The first time I met Patrick, he had to explain Stripe to me. I was, first of all, it was so complicated. Because it’s complicated.

PATRICK COLLISON: We tried to refine the descriptions over time. But you got an early version.

JENSEN HUANG: No, you’re in a complicated business no matter what. But nonetheless, I was so inspired by it. Incredible what you guys have built.

PATRICK COLLISON: Are we going to get you migrated to Stripe Billing now that we have usage-based billing?

JENSEN HUANG: I wish I had a business that required billing.

PATRICK COLLISON: I think your public filing suggests you’re doing a lot of billing. We’ll follow up on it. All right, so.

JENSEN HUANG: It’s only 10 transactions, just so you know. Your economics serving us is like nothing. It’s like 10 transactions.

PATRICK COLLISON: Remember, we’d happily take the 2.9%, but anyway. We can discuss that separately. So...

JENSEN HUANG: Done.

PATRICK COLLISON: Think about this token. You can’t say that. You’re a public company. So thinking about these... token factories, I feel like a big question right now is whether the models saturate in the sense that, you know we demoed the Sigma Assistant on stage earlier. And you can write some natural language and we convert that to SQL. And going from, you know, maybe a 7 billion parameter model to a 70 billion parameter model or something like that, there might be a significant kind of... consequential improvement in query accuracy for the user for the typical kind of queries that people tend to construct. But, maybe going to a model of 10x larger than that is sort of unnecessary. Like, at some point, you get too good enough you can reliably convert the natural language in SQL.

I think there’s a question of... for the use cases for which LLMs are being deployed, what does that saturation curve look like and for how many use cases does one need a trillion-parameter model or a 10 trillion-parameter model? Or do we simply reach a point where some number that is say less than 100 billion is sufficient? Do you have any point of view on that? Or is that even... a reasonable way to look at the question in the first place?

JENSEN HUANG: Okay, let’s break it down. Let’s reason about it.

PATRICK COLLISON: In public appropriately.

JENSEN HUANG: In almost everything every question I get, let’s break it down, let’s reason about it. So let’s start with an example. In 2012, AlexNet was computer vision ImageNet, image recognition, 82% or something like that, accuracy. Over the next... almost not quite 10 years, I think it was like 7 years, every single year, the accuracy error reduced in half. Every year, the error reduced in half, or otherwise known as Moore’s Law. Okay? So you doubled the performance, you double the accuracy, and you double its believability every single year. Over the course of seven years, it’s now superhuman.

Same thing with speech recognition. Same things with natural language understanding. We want to know, we want to believe, not know. We want to believe that the answer that’s being predicted to us is accurate. We want to believe that. And so the industry is going to chase that believability or that accuracy and double its accuracy 2x every year. I believe that’s going to be the same thing with natural language understanding.

And, of course, the problem space is a lot more complicated. But I have every certainty that we’re going to double its accuracy every single year to the point where it is so accurate. We’ve largely tested across many of your examples when you interact with it that you go, “You know what? This is really, really good. I believe the answer that it’s producing for me.” That condition is very important.

The second thing is this: today’s language models, today’s AI and everything that we’ve shown are one shot. And yet, you and I both know that there are many things that we think about that are not one shot. You have to iterate. So how do you come up, how do you reason about a plan? How do you come up with a strategy to solve a problem?

Maybe you need to use tools. Maybe you have to look up some proprietary data. Maybe you have to do some research, in fact. Maybe you have to ask another agent. Maybe you have another, ask another AI. Maybe you have to be a human in the loop, ask a human. Trigger an event, send an email to somebody or text to somebody, get a response before you can move on to the next step of that plan.

And so, a large language model has to iterate and think of a plan. That’s not a one-shot thing. And once it comes up with a plan, as it traverses that graph there’s a whole bunch of language models that are going to get instantiated and initiated. So I think your future models are going to iterate. So instead of a one-shot model, it’s going to be a planning model with a whole bunch of other models around it that are particularly good at particular skills. And so, I think we have long ways to go.

PATRICK COLLISON: Meta garnered a lot of attention last week for the release of Llama 3, which seems to be the most impressive open-source model thus far. Any thoughts on open-source models?

JENSEN HUANG: If you ask me what are the top most important events in the last couple of years, I would tell you, of course, ChatGPT reinforcement learning, human feedback, grounding into human values and having the technology necessary to do that, obviously a breakthrough and democratized computing. It made it possible for everybody to be a programmer. Everybody’s now doing amazing things with it. ChatGPT, the work that OpenAI did. Greg and Sam and the team, really proud of them.

The second thing that I would say, that is just as important, is Llama, not Llama 1, but Llama 2. Llama 2 activated just about every industry to jump into working on generative AI. And it opened the floodgates of every industry being able to access this technology. Health care, financial services, you name it, manufacturing, you name it, customer service, retail, all kinds. I think Llama 2 and Llama 3, because it’s open sourced, it engaged research and engaged startups, engaged industry. It made generative AI accessible. I think that’s a very big deal.

And so, I think ChatGPT democratized computing. I think Llama democratized generative AI. Does that make sense? And I think without it, it’s very hard to have activated all of the research on safety and all of the different ways of chains of thoughts and all the reasoning technology that’s now being developed and all the reinforcement learning stuff. That stuff would’ve been very hard to have activated without Llama.

PATRICK COLLISON: Dario Amodei was on Ezra Klein’s podcast two weeks ago and he, as many others have, many others in particular who are... involved with Frontier Labs, was predicting AGI in the relatively near term, conceivably the next couple of years, years like 2027 and so on are frequently thrown around. Thoughts?

JENSEN HUANG: Depending on how you define AGI. Now, first of all, as an engineer you know that we can only solve a problem ultimately if you can measure it. And so, you have to express the problem statement, the mission somehow in some measurable way. If you told me that AGI is the list of benchmarks we currently use, they’re math tests and English comprehension tests and reasoning tests, and you know. You got medical exams and bars.

You make your list of all of the tests that you want. It doesn’t matter what it is. Just make your list. If you make your list, I am certain we will achieve excellent results in a very nominal amount of time. And if that’s the definition of AGI, I’ll make a guess it’s probably, definitely, within the next five years. So all of the tests that we currently measure these models with their accuracy or their error rate is reducing in half every six months. So there’s no reason why we shouldn’t expect it all to be superhuman pretty soon.

PATRICK COLLISON: So again, everyone in this audience...

JENSEN HUANG: But that doesn’t meet the standard. Just be clear. That doesn’t meet the standard of a normal person thinking it’s AGI. Does that make sense? A on-the-street person, hey, AGI, that’s probably not what they’re thinking what I defined it as. The way I defined it is simply an engineering way of defining it so that you can answer that question. The second way of answering the question is when can you achieve AGI in an undefined way? If it’s undefinable, then how long would you know... How long would it take? Undefinable.

PATRICK COLLISON: So everyone in this audience, again, runs a business. And so, a practical question they/we all face is how do you know if you are in the face of the kinds of changes you just depicted, how does one know, how can one know whether one is responding appropriately sufficiently in the right ways, etcetera? Any advice?

JENSEN HUANG: If you’re not engaging AI actively and aggressively, you’re doing it wrong. You’re not going to lose your job to AI. You’re going to lose your job to somebody who uses AI. Your company is not going to go out of business because of AI. Your company is going to go out of business because another company used AI. There’s no question about that.

And so, you have to engage AI as quickly as possible. You have to engage AI as quickly as possible so that you could do things that you think cost too much to do. For example, if the marginal cost of intelligence was practically zero, there are a lot of things that you would do now that you wouldn’t have done otherwise. And so, notice how often we do search and these days, notice how often we ask questions. I mean, any random question I’ll be asking Perplexity, right? And so, why not? Just in case.

PATRICK COLLISON: Aravind just gave a talk here at Sessions.

JENSEN HUANG: Okay. I love using it. And even if I know the answer, I’ll just ask it anyways. You know, just to see what it comes up with. And so I think we want that to happen. We want the marginal cost of these type of activities to be as low as possible so that you use it in abundance. Second, if you could use AI to be productive, you know that productive companies leads to higher earnings. Higher earnings leads to more employment. More employment leads to more social growth. So there is a lot of reasons to want to drive productivity into companies.

PATRICK COLLISON: And apart from just changing your manufacturing plans and your CapEx plans, how has AI changed how NVIDIA works internally?

JENSEN HUANG: We were one of the first technology companies to invest in our own AI supercomputers. We can’t design a chip anymore without AI. At night, our AIs are exploring design spaces vast and wide that we would never do ourselves because it costs too much money to explore it. And so we... I... Our chips are so much better. Because of an AI, we could reduce the amount of energy used for our chips as higher performance. Our software, we can’t write software without AI anymore. We have to explore all the... The design space of optimizing compilers is too large.

We use AIs to file bugs. So our bugs database actually tells you who’s... what’s wrong with the code, who’s likely involved and activates that person to go fix it, you know? And so, I think I want everybody, every organization in our company to use AI very aggressively. I want to turn NVIDIA into one giant AI. How great would that be? And then, I’ll have work-life balance.

PATRICK COLLISON: Are there any favorite examples you’ve heard of businesses and maybe in some kind of unexpected sector or some unexpected use case where you feel they kind of can serve as a poster child for some of the dynamics you’re describing where they’ve really realized some of this opportunity?

JENSEN HUANG: Well, the biggest surprise of AI that shouldn’t be a surprise for a lot of people is that when we say, “It’s a large language model,” the word language doesn’t mean human language only. And it doesn’t mean English only or French only or Irish only, that’s a whole different language but... Is there a large language model for Irish?

PATRICK COLLISON: I’ve tried it.

JENSEN HUANG: That works?

PATRICK COLLISON: Yeah. It works well. John and I spent most of our education in Ireland being taught in Irish. So these models are some of the first people I’ve had the chance to have a dialogue with as Gaeilge, in...

JENSEN HUANG: Very surprising.

PATRICK COLLISON: Many years. And actually I’ve been enjoying… have you played with Suno?

JENSEN HUANG: Suno?

PATRICK COLLISON: Suno. Suno is an app for creating music. Synthetic music. Okay. And I’ve been enjoying creating...

JENSEN HUANG: Irish music.

PATRICK COLLISON: I, of course, tested it on that. And Celtic dubstep is a thing that it can do.

JENSEN HUANG: Fantastic. Okay. Makes sense. Like, if it could do that, then of course they could learn the language of life. Of course, they could learn and if a language model can understand sound, which is a sequence time series, it’s a sequence, why can’t it learn robotics articulation, which is a sequence? You just have to figure out how to tokenize it. So the idea that all of a sudden “Oh hey, look, listen, I could also learn SQL. I could learn ABAP, I could learn Lightning. I could learn all these proprietary languages. I could learn Verilog,” I could learn, right? So, all of a sudden, you realized, hang on a second, I can put a Copilot on top of every tool on the planet.

PATRICK COLLISON: Well, and to this point, and you know, NVIDIA being one big AI is the future one of 100,000 models or 100 million models or is the future one of one model and there’s just like a model that does all the things.

JENSEN HUANG: I think that it would be great to have... It would be great to have supermodels that help you reason about things in general, but... For us, for all companies that have very specific, domain-specific expertise, we’re going to have to train our own models. And the reason for that is because we have a proprietary language. That difference between 99% and 99.3% is the difference between life and death for us. So it’s too valuable to us. No different than fraud detection for you.

PATRICK COLLISON: I was going to say...

JENSEN HUANG: It’s too important to you.

PATRICK COLLISON: That’s been exactly our experience.

JENSEN HUANG: Yeah. It’s too important to you. However good the general model is, you’re going to want to take that and fine-tune it and improve it into perfection because it’s just too important to you.

PATRICK COLLISON: So we’re going to shortly run out of time here and there’s a whole bunch of questions I haven’t gotten to yet. I’ve exercised poor discipline on the time management front. So there’s a bunch that I think are... I was told I definitely had to ask you but there’s a couple that I really wanted to ask and it’s only us up here, so. Lisa Su is your first cousin once removed?

JENSEN HUANG: Yes. She’s terrific. She’s amazing.

PATRICK COLLISON: And then, AMD is now...

JENSEN HUANG: She’s the CEO of AMD, by the way.

PATRICK COLLISON: Yeah. And AMD is now one of your competitors in the GPU space?

JENSEN HUANG: No. We’re family.

PATRICK COLLISON: Okay.

JENSEN HUANG: We’re all in the industry.

PATRICK COLLISON: One of your partners in the industry.

JENSEN HUANG: Yeah. Yeah. We buy from AMD.

PATRICK COLLISON: What’s going on in the water? How did we end up with two of the... arguably the two most important GPU companies being run by close relatives? What’s going on?

JENSEN HUANG: You got to keep it close to the family. No, I just, I have no idea how it happened. We didn’t grow up together and we didn’t know each other.

PATRICK COLLISON: That makes it even more interesting, right?

JENSEN HUANG: Yeah. We didn’t even know each other until she was at IBM. And her career is incredible. She’s really quite extraordinary, yeah. I think this question requires further study. Yeah.

PATRICK COLLISON: So. Okay, you’ve been operating in Silicon Valley since the early ’90s.

JENSEN HUANG: Yes.

PATRICK COLLISON: How has Silicon Valley culture changed in that time?

JENSEN HUANG: Oh, wow. I haven’t thought about this in a long time. I guess in a lot of ways, in a lot of ways, probably... Okay, here’s one. When I first started NVIDIA. I was 29 years old and... I was 29 years old with acne. And... you go talk to your... go recruit law firms and VCs and you know, I got a big zit on my forehead. I don’t have one today so I feel comfortable talking about it. But it could happen.

And so anyways you feel rather insecure, because most of CEOs back then wore suits and they’re quite accomplished, and they sound like adults and they use big words and they talk about business and things like that. So when you’re young, you feel rather intimidated. You’re surrounded by a bunch of adults.

Well, you know, now, if you don’t have acne, I don’t think you deserve to start a company. I just, that’s one big difference. Acne. You know, the takeaway from Jensen’s speech. I just, what it means is really... we’ve enabled younger people to be extraordinary. I think that the young generation of CEOs the type of things that you guys know at such a young age is really quite extraordinary. I mean, it took me decades to learn it, and so.

PATRICK COLLISON: Last question.

JENSEN HUANG: That was a compliment. See how he quickly, he changed it? I wasn’t saying you have acne. I was just saying you were smart.

PATRICK COLLISON: NVIDIA has a market cap of roughly $2 trillion dollars and you’re... and you’re now within spitting distance of Apple and Microsoft. And I just checked, and they have 220,000 and 160,000 employees respectively. NVIDIA has 28,000 employees. So, you know, less than a fifth of the smaller of the two there.

And then, you just said, when we were chatting backstage and I jotted this down. “You can achieve operational excellence through process but craft can only be achieved with tenure.” And so, NVIDIA is considerably smaller than any of the other giants. And you seem to think that tenure really matters and I guess that craft really matters. Say a little bit more there.

JENSEN HUANG: The, I think, extraordinary thing... I think a lot of good things could be made. Good things are made with operational excellence. But you can’t make extraordinary things through just operational excellence. And the reason for that is because a lot of the great things in your body of work and the products that you make, the company you created, the organizations you’ve nurtured, it takes loving care. And you can’t even put it in words. How do you put loving care in an email? And for people to go, “Oh, I know exactly what to do.” You can’t put that... you can’t put that in a business process, loving care, and...

PATRICK COLLISON: Is love and care kind of an NVIDIA catchphrase?

JENSEN HUANG: Well, I use love fairly abundantly and care, I use abundantly.

PATRICK COLLISON: At Stripe, we talk a lot about craft and beauty.

JENSEN HUANG: Yeah, right. You have to use these words because, that in a lot of ways, there are no other words to describe it. You can’t put it in numbers. You can’t write it in the product specification. The product specification says, “I want you to build something great that’s incredibly beautiful, in great, great craft.” You can’t specify these things, and so.

PATRICK COLLISON: But I’m sure there is people at Stripe who think, “Patrick’s always yammering on about craft and beauty, and it’s this kind of...”

JENSEN HUANG: I never yammer. I just want to let you know that. I don’t even know what that sounds like.

PATRICK COLLISON: Okay, well, yeah.

JENSEN HUANG: Yammering on. Go ahead. Go ahead.

PATRICK COLLISON: Yeah, you’re more lucid than I am. I just babble. But hey, so Patrick is always going on about this craft and beauty stuff and wants things to have this particular ineffable character but it doesn’t directly serve some customer need and so forth. Like, customers aren’t coming to us and saying, “I want the product to be more beautiful.” They’re saying, “I want it to feature X or feature Y.” And yet, we believe that the craft and beauty really matters. It sounds like you’re getting at something similar. Why do you think it matters?

JENSEN HUANG: Actually, your customers, even though they didn’t say it they might not have the words to say it, but when they experience it, they know it. There’s no question. Look, I... Look, Stripe’s work has beauty, has elegance, has simplicity. Simplicity is not simple as you guys know. Simplicity and simple are not the same thing. And it has elegance and it solves the problem but just enough. It burdens you, but not too much. You know?

And so that... and that balance is hard to find. And you can’t specify that. You just feel your way there. And when you have a team that’s with you that feels the way they’re together, in a lot of ways we’ve codified, we’ve encoded the magic of the company in a way that no words can describe. And you don’t want to lose that. You don’t want to lose that. You want to take that and take it to the next level next time.

And so I don’t want to reset. I don’t like working with new people for that reason because I’ve encoded, I’ve embodied, I’ve deposited so much pain, suffering, joy, knowledge, right? All that experience, life experience you’ve encoded it in, all the people that you’ve worked with. You want to carry it on. You want to take it to the next level. And that’s really the reason why I really deeply believe in tenure. And because of that, small teams could do great things.

And NVIDIA is kind of a small team, we’re 28,000 people. People think we punch well above our weight because of that reason. And so, it’s amazing what you guys have done and how incredibly small you are, 7,000 people supporting a trillion dollars worth of ecosystem and industry and... and economy, and who knows how far you guys can go. So, I’m very proud of you.

PATRICK COLLISON: Jensen, thank you.