Jen-Hsun Huang's latest 10,000-word interview: AGI is coming, AI will revolutionize productivity

Large-scale distributed computing is an important direction for future AI development

Brad Gerstner:

One end is right. Musk is on one end. Yes, you, but you answered that question by starting off by saying, yes, there are 200 to 300,000 GPU clusters here. Yes, right. Can this scale to 500,000? Can it scale to a million? Do your product requirements depend on it scaling to 2 million?

Huang Jen-hsun (1944-), Chinese-American physicist::

The last part is a negative.My feeling is that distributed training has to work. My feeling is that distributed computing will be invented. Some form of federated learning and distributed computing, asynchronous distributed computing will be discovered.

I'm very enthusiastic and optimistic about this, but of course, the thing to realize is that the scaling laws used to be about pre-training. Now we've moved to multimodality, we've moved to synthetic data generation, and post-training has now scaled incredibly well. Synthetic data generation, reward systems, reinforcement based learning, and then now inference scaling has peaked. A model has done an incredible 10,000 internal inference before it answers your answer.

This may not be unreasonable. It may have completed a tree search. It may have done reinforcement learning based on that. It may, it may have done some simulations, certainly done a lot of reflection, probably looked up some data, looked at some information, didn't it? So he's probably got quite a lot of background. I mean, this type of intelligence is. Well, that's what we did. That's what we've done. Isn't it? So for the capability, this extension, I just did the math and compounded it with the model size and the computational size four times per year.

On the other hand.Demand continues to grow in terms of usage.Do we think we need millions of GPUs? No doubt. Yes, now that's a yes. So the question is, how do we build it from a data center perspective? A lot of it has to do with whether the data center is a few gigawatts at a time or 250 megawatts at a time. My sense is that you're going to get both.

Clark Tang:

I think analysts always focus on current architecture bets, but I think one of the biggest takeaways from this conversation is that you're thinking about the entire ecosystem and many years into the future. So, because NVIDIA is just scaling up or scaling out, it's to meet future demand. That's not to say that you can only rely on a world with 500,000 or even a million GPU clusters. When distributed training comes along, you write software to implement it.

Jen-Hsun Huang:

We developed Megatron seven years ago, and yes, scaling of these large training tasks happens. So we invented Megatron, so all the model parallelism that's going on, all the breakthroughs in distributed training and all the batch processing and all that stuff is because we did the early work and now we're doing the early work for the next generation.

AI changes the way work is done

Brad Gerstner:

So let's talk about Strawberry and o1. i think it's cool that they named it after o1. It means recruiting the best and brightest people in the world and bringing them to America. I know we're all passionate about that. So I love the idea of building a model of thinking that will take us to the next level of extended intelligence, right, and it's a tribute to the fact that it's these people who came to America through immigration that made us what we are, that brought their collective intelligence to America.

Jen-Hsun Huang:

Of course. And alien intelligence.

Brad Gerstner:

Of course. This was spearheaded by our friend Noam Brown. Reasoning about how important temporal reasoning is as a whole new vehicle for extending intelligence is separate from just building bigger models.

Jen-Hsun Huang:

It's a big deal. It's a big deal. I think a lot of intelligence can't be done a priori. Right. A lot of computations, even a lot of computations can't be reordered. I mean, unordered execution can be done a priori, and a lot of things can only be done at runtime.

So, whether you're thinking about it from a computer science perspective or from an intelligence perspective, too many things need context. Context, right. And quality, the type of answer you're looking for. Sometimes a quick answer is enough. It depends on the consequences of the answer, the impact. It depends on the nature of the use of the answer. So, some answers, please take an evening, some answers take a week.

Yes. Right? So I can totally imagine me sending a prompt to my AI telling it to think about it for a night. Think about it all night. Don't tell me right away. I want you to think about it all night and then tell me tomorrow. What's your best answer and reasoning for me. So, I think from a product standpoint, the quality now, the smart segmentation. There will be one-off versions. Absolutely. And some that take five minutes.

Right? And humans. So we're going to be a huge employee base, if you will. Some of them are in AI.digital person, some of which are biological humans, and I hope some of which are even super robots.

Brad Gerstner:

I think that's a grossly misunderstood thing from a business perspective. You've just described a company that has the equivalent amount of output of a company with 150,000 people, but you've done it with only 50,000 people. That's right. Now, you're not saying I'm going to fire all my employees. No. You're still increasing the number of employees in the organization, but the amount of output from that organization will increase dramatically.

Jen-Hsun Huang:

This, this is often misunderstood. ai is not me. ai is not going to change every job. ai is going to have a huge impact on the way people work. Let's recognize that.AI has the potential to do incredible good. It also has the potential to cause harm. We must build safe AI.Yes, let's lay that foundation. YES. All right.

Jen-Hsun Huang:

The part that people overlook is that when a company uses AI to improve productivity, it will likely show up as better earnings or better growth, or both. When that happens, the CEO's next email will likely not be about layoffs.

Brad Gerstner:

Of course it's an announcement, because you're growing.

Jen-Hsun Huang:

The reason is that we have more ideas that we can explore and we need people to help us think carefully before we automate. So the automation part, AI can help us do that. Obviously, it will also help us think about it, but it still requires us to figure out what problem am I trying to solve. There are trillions of problems that we can solve. So what are the problems that companies need to solve, pick those ideas and figure out ways to automate and scale them. So as we become more productive, we're going to hire more people. People forget that, and if you go back in time, obviously we have more ideas today than we did 200 years ago. That's why the GDP is bigger and more people are employed. Even though we're automating like crazy at the bottom.

Brad Gerstner:

It's a very important point in this period that we're entering a period where almost all human productivity, almost all human prosperity is a byproduct of automation. The technology of the last 200 years. I mean, you can look at the creative destruction of Adam Smith and Shem Peter, you can look at the graph of GDP per capita growth over the last 200 years, and now it's accelerating.

Yeah, that got me thinking about this. If you look at the 90s, we had productivity growth in the U.S. of about 2.5% to 3% per year, okay? And then in 2010, it slowed down to about 1.8%. and then the last 10 years have been the slowest decade for productivity growth. So that's our fixed amount of labor and capital or amount of output that's actually the slowest on record.

A lot of people are debating the reason for this. But if the world is really as you describe, and we are going to utilize and make intelligence, then aren't we on the verge of a dramatic expansion of human productivity?

Jen-Hsun Huang:

It's our hope. This is our hope. Of course, we live in this world, so we have direct evidence.

We have direct evidence of either isolated cases or individual researchers who are able to utilize AI to explore science at unimaginable mega-scales. That's productivity. One hundred percent measures productivity, or the fact that we're designing such incredible chips at such a high rate. The complexity of the chips we're building and the complexity of the computers we're building is growing exponentially, and a company's employee base is not a measure of productivity, right.

The software we develop is getting better and better because we use AI and supercomputers to help us. The number of employees has grown almost linearly. Another reflection of productivity.

So, I can delve into that and I can sample a lot of different industries. I can check it out myself. Yeah, you're right. Commerce. That's right.

So I can, of course, you can't, we can't, we might overfit. But the art of it is of course to generalize what it is that we're observing and whether that's going to be reflected in other industries.

There is no doubt that AI is the most valuable commodity the world has ever known. Now we have to mass produce it. We, we, all of us have to be good at what happens if you're surrounded by these AIs that are doing very well, much better than you. When I look back, this was my life. I had 60 direct reports.

They're world-class in their field, and they do it better than me. Much better than me. I have no trouble interacting with them, and I can design them effortlessly. I can program them effortlessly as well. So I think what people have to learn is that they're all going to be CEOs.

They're all going to be CEOs of AI agents. they have the ability to have the creativity, well, some knowledge, and how to reason, how to break down problems, so that you can program these AIs to help you achieve the same goals that I do. That's running a company.

AI Security Requires a Multi-Party Effort

Brad Gerstner:

Now. You mentioned something about uncoordinated, secure AI. you mentioned the tragedy that's happening in the Middle East. We have a lot of autonomy and a lot of AI being used around the world. So let's talk about the bad guys, secure AI, coordination with Washington. How are you feeling today? Are we on the right path? Do we have an adequate level of coordination? I think Mark Zuckerberg has said that the way we defeat bad AI is by making good AI better. How would you characterize your view of how we can make sure that this has a positive net benefit for humanity rather than this anti-utopian world that we're living in.

Jen-Hsun Huang:

The discussion about security is really important and good. Yes, the abstract view, the conceptual view of AI as a giant network of neurons, is not so good, right. Good. The reason for that is that it's well known that AI and large language models are related, not the same thing. I think there's a lot of things that are being done that are very good.First, open-source models so that the entire research community, every industry and every company can participate in AI, theYes, and learn how to utilize that ability for applications. Very good.

Second, people underestimate the number of technologies dedicated to inventing AI to keep it safe.Yes, AI can organize data, carry information, train, create AI to coordinate AI, generate synthetic data to expand AI's knowledge and make it less illusory. All AI systems that are created for vectorization or graphing or any other AI system that is used to inform the AI, protect the AI to monitor other AIs, the secure AI created by these AI systems is being praised, right?

Brad Gerstner:

Then we've established it.

Jen-Hsun Huang:

That. We're building it all. Yes, across the industry, the methodologies, the red teams, the processes, the model cards, the evaluation systems, the benchmarking systems, all of that, all of that is being built at an unbelievable rate in the harness. I want to know, celebrate. Do you guys understand? Yes, you do.

Brad Gerstner:

And, no, there's no, there's no government regulation that says you have to do this. Yes, the players building these AIs in the space today are taking these key issues seriously and harmonizing around best practices. That's right.

Jen-Hsun Huang:

So this has not been fully appreciated or understood. It is. There needs to be somebody, there needs to be, everybody needs to start talking about AI, which is an AI system, which is an engineered system, which is carefully designed, built from first principles, fully tested, and so on. Remember, AI is a capability that can be applied. I don't think it's necessary to regulate important technologies, but it's also important not to over-regulate to the point where some regulation is done for most applications. All of the different ecosystems that already regulate technology applications must now regulate technology applications that now incorporate AI.

So, I think it's important not to misunderstand and ignore the vast amount of regulations that have to be initiated in the world for AI. Don't rely on just one cosmic galaxy. the AI Council may be able to do that because all of these different agencies are in place for a reason. All of these different regulatory bodies were created for a reason. Going back to the original principle, I would.

Open source vs. no open source is a false dichotomy

Brad Gerstner:

You have launched a very important, very large, very powerful open source model.

Jen-Hsun Huang:

Nemotron.

Brad Gerstner:

Yes, it's clear that Meta has made a significant contribution to open source. I find when I read Twitter, there's a lot of discussion about open vs. closed. How do you view open source, your own open source model, and can you keep up with the cutting edge? That's the first question. The second question is, you know, having open-source models and closed-source models, which power commercial operations, is that your view of the future? Do those two things create a healthy tension for security?

Jen-Hsun Huang:

Open source and closed source are related to security, but not only security. For example, there is absolutely nothing wrong with having closed source models, they are the engine that sustains the economic modeling necessary for innovation. Well, I totally agree with that.I think the dichotomy of closed versus open is wrong.

Because openness is necessary for many industries to be activated, now, if we didn't have open source, how could all of these different fields of science be activated, activate AI. because they have to develop their own domain-specific AI, they have to develop their own AI using open source models to create domain-specific AI. they're related, they're not, again, not the same. Just because you have an open source model doesn't mean you have AI. so you have to have that open source model to create AI. so financial services, healthcare, transportation, the list of industries, fields of science that have now been enabled because of open source.

Brad Gerstner:

Unbelievable. Do you see much demand for your open source model?

Jen-Hsun Huang:

Our open source model? First. llama download. Obviously, yes, Mark and the work they've done is incredible. Beyond belief. It is. It completely activates and engages every industry, every field of science.

Okay, sure. The reason we did the Nemotron was to generate synthetic data. Intuitively, an AI would somehow sit there and loop and generate data to learn about itself. That sounds fragile. It's doubtful how many times you can go around this infinite loop, this loop. However, the picture I have in my head is kind of like if you take someone super smart, put them in a padded room, close the door for about a month, and what comes out might not be a smarter person. So, so, but you can have two or three people sitting together, we have different AIs, we have different distributions of knowledge, and we can go back and forth for quality assurance. All three of us can get smarter.

So the idea that you could have AI models swapping, interacting, passing back and forth, discussing reinforcement learning, synthetic data generation, etc., makes intuitive sense to suggest and make sense.Therefore, our model Nemotron 350B is is the best reward system model in the world. Therefore, it is best criticized.

Interesting. It's a great model to enhance other people's models. So no matter how good someone else's model is, I would recommend using the Nemotron 340B to enhance and improve it. We have seen Llama get better and make all other models better.

Brad Gerstner:

As someone who took delivery of a DGX1 in 2016, it really has been an incredible journey. Your journey has been both incredible and unbelievable. It's as remarkable as just surviving the early days. You delivered the first DGX1 in 2016, and we have a Cambrian moment in 2022.

So I'm going to ask you a question I've often wanted answered, and that is, how long can you maintain your current job with 60 direct reports? You are everywhere. You are driving the revolution. Are you having fun? Is there anything else you'd rather be doing?

Jen-Hsun Huang:

It's about the last hour and a half.The answer is "I enjoyed it".Great times. I can't imagine anything else I'd rather be doing. Let's see. I don't think it's right to give the impression that our work is always fun. My work isn't always fun, and I don't expect it to always be fun. Do I ever expect it to always be fun? I think it's always important.

Yes, I don't take myself too seriously. I take my job very seriously. I take our responsibilities very seriously. I take our contributions and our moments very seriously.

Is it always fun? No, it's not. But have I always enjoyed it? Yes. Like everything, whether it's family, friends or kids. Is it always fun? No, it wasn't. Did we always enjoy it? Absolutely.

So I think, me, how long can I do this? The real question is, how long can I stay relevant? That's what matters, and the answer to that question can only be how will I continue to learn? Today I am more optimistic. And I'm not just saying that because of our topic today. I'm more optimistic about my ability to say relevance and continue to learn because AI. I use it every day, I don't know, but I'm sure you all do. I use it almost every day.

There isn't a single piece of research I do that doesn't involve AI. and yes, there isn't a single question that I double-check with AI even if I know the answer. Yes, surprisingly, the next two or three questions I asked revealed something I didn't know. You choose your topic. You choose your subject. I consider the AI a mentor.

AI is an assistant, AI is a partner that can brainstorm with me and check in with me, and guys, it's totally revolutionary. I am an information worker. I output information. So I think their contribution to society is amazing. So I think that if that's the case, if I can maintain that relevance and continue to contribute, and I know that this work is important enough, yes, I want to continue to pursue it, I have an incredible quality of life. So I will.

Brad Gerstner:

Say I can't imagine missing this moment when you and I have been working in this field for decades. This is the most important moment of our careers. We are very grateful for this partnership.

Jen-Hsun Huang:

Nostalgia for the next decade.

Brad Gerstner:

Partnership of ideas. Yes, you make things smarter. Thank you. I think it's really important for you to be part of the leadership, right, that will optimistically and safely lead this forward. So thank you.

Jen-Hsun Huang:

With you guys. Really happy. Really. Thanks for that.