Early days of AI (and AI Hype Cycle)

Raw Text

Rather then view LLMs, Transformers, and diffusion models as part of a continuum with past "AI", it is worth thinking of this as an entirely new era and discontinuity from the past

Elad Gil

52

Share

I worked on early ML systems and products at Google and later at Twitter (after they bought my company, Mixer Labs). I then spent a decade working as a founder and executive & investing in machine learning companies. Until the rise of new AI architectures (in particular transformer-based and diffusion-model based approaches), roughly all machine learning startups failed. Value in prior AI waves went largely to incumbents over startups - as the capabilities were not advanced enough to create new market openings.

Here is a slide I used to use (borrowed from Brandon Ballinger ) during 2017-2019 or so - this slide reflected the CNN/RNN/GAN world of the prior ML wave.

Thanks for reading Elad Blog! Subscribe for free to receive new posts and support my work.

Subscribe

When many business people talk about “AI” today, they treat it as a continuum with past capabilities of the CNN/RNN/GAN world. In reality it is a step function in new capabilities and products enabled, and marks the dawn of a new era of tech.

It is almost like cars existed, and someone invented an airplane and said “an airplane is just another kind of car - but with wings” - instead of mentioning all the new use cases and impact to travel, logistics, defense, and other areas. The era of aviation would have kicked off, not the “era of even faster cars”.

(We should of course, fully recognize how important prior waves of ML and deep learning were and are to all this - however, to treat it as an ongoing continuum may miss the seismic nature of this technology shift).

Slide I now use.

The biggest inklings that something interesting was afoot came kicked with GPT-3 launching in June 2020. GPT-3 was a massive step up from GPT-2 and prior models. It was not quite good enough to do all the things we now view as hallmarks of “AI”, but it was highly suggestive of what was to come (I went on the A16Z podcast a few months later to talk about GPT-3, as it was so striking). For those in the know, the launch of GPT-3.5 in March 2022 solidified the perception of transformer-based models as the future. Internally at companies like Google, OpenAI, Microsoft, and Anthropic, early access to models gave a subset of people a glimpse of the future that was coming. This led to a Google engineer eventually proclaiming an internal AI chatbot named LaMDA as being “sentient” - this chatbot was a sort of predecessor to chatGPT and products like Character.AI.

The real starting guns for this AI wave in terms of a large number of founders jumping in was driven by two sets of launches. First were the launches of image-gen products like Midjourney and Stable Diffusion, followed a few months later by ChatGPT, which wowed the world, captured the public imagination, and was the AI startup big bang moment. ChatGPT truly highlighted the capabilities of these new forms of AI and the power of RLHF. OpenAI followed up with GPT-4, 4 months later.

True enterprise adoption is still many quarters/years away

ChatGPT’s launch was the starting gun for mainstreaming that AI is a big deal in terms of new capabilities and kicked off the large scale enthusiasm, hype cycle, and adoption for generative AI. This launch was only 8-9 months ago, and GPT-4 did not come out until 5 months ago. Given that large enterprise planning cycles often take 3-6 months, and then prototyping and building will take a year for a large company, we are still very far away from peak AI usage or peak AI hype. Most large enterprises are still trying to analytically sort what “AI” means for them, and are still many quarters from embracing this new technology.

4 Waves of AI Adoption

Indeed, there are likely at least 4 waves of AI to consider in these early days.

Wave 1: GenAI native companies . ChatGPT, Midjourney, Character.AI, Stable Diffusion, Github copilot, and other early launches that have now gained significant revenue and user traction. Obviously there are some great ML companies that pre-date GenAI that continue to participate in the current era (Hugging Face, Runway, Scale, WandB are a few that come to mind).

Wave 2 (current wave): Early startup adopters and fast mid-market incumbents. This is the first wave of startups to launch on top of GPT-3.5/4 like Perplexity, Langchain, Harvey or others. In parallel, a small number of founder led multi-billion companies like Navan, Notion, Quora, Replit, and Zapier launched AI-powered products quickly and are the early adopters of the wave. Microsoft, Adobe, and Google are notable outliers as very large enterprise moving fast to AI - Microsoft likely due to its inside track with OpenAI, and Adobe as diffusion models tend to be cheaper and simpler to train than the large scale LLMs.

Wave 3 (coming soon): Next wave of startups currently being founded. It will be exciting to see what is in this mix and may include new formats like voice and video in addition to using natural language in more verticals and more ways, as well as new types of infrastructure. Companies like Eleven Labs/ LMNT /LFG Labs, Braintrust , and many more will provide incremental experiences. There is a big wave of new startups coming. The current YC batch alone appears to have a 100 or more AI startups….

Wave 4 (coming 2024/2025?): First big enterprise adopters. Since enterprise planning and build cycles are so long, anticipate the first really products (versus demos or prototypes) from larger companies other than MSFT, Adobe, Google, Meta to start to show up in a year or two. This is when revenue to AI infra companies will start to ramp significantly relative to today, when hype will peak, and we will see further accelerated investment in AI.

The Future Is Bright

There is enormous potential for this new wave of tech to impact humanity. For example, Google’s MedPaLM2 model outperforms human physicians to such a strong degree that having medical experts RLHF the model makes it worse (!).

Given the strong potential it will be exciting to see all the immense innovation in education, healthcare, enterprise and consumer software, and other aspects of life coming via this tech breakthrough.

We are only 8-9 months since chatGPT woke the world to this new era of AI, and exciting times are ahead as we follow this tech discontinuity down our timeline. It is the very earliest days of AI, and both peak hype and peak impact are still in the future. Lots more is still to come.

MY BOOK You can  order the High Growth Handbook here . Or  read it online for free .

OTHER POSTS

Firesides & Podcasts

Clem Delangue: Hugging Face, Open Source, AI

Dylan Field: Figma, AI & Design, Education

Reid Hoffman on AI, Big Tech, and Society

Sam Altman, CEO OpenAI

Emad Mostaque, Stability.AI

NoPriors AI Podcast

Markets:

AI Safety: Technology vs Species Threats

Startup Decoupling and Reckoning

Defensibility and Competition

AI Platforms, Markets, and Open Source

Changing times (or, why is every layoff 10-15%?)

AI Startup Vs Incumbent Value

AI Revolution - Transformers and Large Language Models

Startup Markets Summer 2022

False Narrative Around Theranos

Index Companies

Defense Tech

Collaborative Enterprise

Industry Towns: Where you start a company matters

Markets are 10X Bigger

Hot Markets 2019

End of Cycle?

Machine Learning Startups

3 Types Of Platform Companies

Defensibility and Lock-In: Uber and Lyft

Uber And Disruption

Who Cares If Its Been Tried Before?

The Road To $5 Billion Is A Long One

How To Win As Second Mover

End Of Silicon Valley

Social Products

Hot Markets For 2015

Startup life

Startups Are An Act of Desperation

Back To The Office

Hiring Executives and Bad Advice

When executives break

Fear of Sales

A brief guide to startup pivots

4 Ways Startups Fail

Founder Investors and Scout Programs

Better Meetings

Magic Startup Moments

Founder Investors & Scout Programs

Jobs, Wozniak, Cook

Co-Founders

How To Choose A Co-Founder

Unequal Cofounders

How To Fire A Co-Founder

Founders Should Divide and Conquer

Raising Money

Preemptive rounds

Don't Ask For Too Much Money

Building VC Relationships

Founders Should Divide And Conquer

Lead VC Vetos

What Is A Good VC?

How To Choose The Right VC For You

Signs a VC Just Isn't That Into You

Series A Crunch

Questions VCs Will Ask You

How To Raise A Successful VC Round

Differences Between Funding Rounds: Series Seed, A, B, C...

Financing Approaches Most Likely To Kill Your Company

Party Rounds: How to Get A High Valuation For Your Seed Startup

20 Questions To Ask Yourself Before Raising Money

The 7 Types Of Angel Investors

Fundraising Will Take You 3 Months

How To Sell Secondary Stock

Old Crypto Stuff:

Core Crypto Use Cases (2018)

The Case For Ethereum

Bitcoin Network Effects

Cryptocurrency Incentives and Corporate Structures

Big Banks and Blockchains

Cryptocurrency's Netscape Moment

Thanks for reading Elad Blog! Subscribe for free to receive new posts and support my work.

Subscribe

52

Share

Previous

Single Line Text

Rather then view LLMs, Transformers, and diffusion models as part of a continuum with past "AI", it is worth thinking of this as an entirely new era and discontinuity from the past. Elad Gil. 52. Share. I worked on early ML systems and products at Google and later at Twitter (after they bought my company, Mixer Labs). I then spent a decade working as a founder and executive & investing in machine learning companies. Until the rise of new AI architectures (in particular transformer-based and diffusion-model based approaches), roughly all machine learning startups failed. Value in prior AI waves went largely to incumbents over startups - as the capabilities were not advanced enough to create new market openings. Here is a slide I used to use (borrowed from Brandon Ballinger ) during 2017-2019 or so - this slide reflected the CNN/RNN/GAN world of the prior ML wave. Thanks for reading Elad Blog! Subscribe for free to receive new posts and support my work. Subscribe. When many business people talk about “AI” today, they treat it as a continuum with past capabilities of the CNN/RNN/GAN world. In reality it is a step function in new capabilities and products enabled, and marks the dawn of a new era of tech. It is almost like cars existed, and someone invented an airplane and said “an airplane is just another kind of car - but with wings” - instead of mentioning all the new use cases and impact to travel, logistics, defense, and other areas. The era of aviation would have kicked off, not the “era of even faster cars”. (We should of course, fully recognize how important prior waves of ML and deep learning were and are to all this - however, to treat it as an ongoing continuum may miss the seismic nature of this technology shift). Slide I now use. The biggest inklings that something interesting was afoot came kicked with GPT-3 launching in June 2020. GPT-3 was a massive step up from GPT-2 and prior models. It was not quite good enough to do all the things we now view as hallmarks of “AI”, but it was highly suggestive of what was to come (I went on the A16Z podcast a few months later to talk about GPT-3, as it was so striking). For those in the know, the launch of GPT-3.5 in March 2022 solidified the perception of transformer-based models as the future. Internally at companies like Google, OpenAI, Microsoft, and Anthropic, early access to models gave a subset of people a glimpse of the future that was coming. This led to a Google engineer eventually proclaiming an internal AI chatbot named LaMDA as being “sentient” - this chatbot was a sort of predecessor to chatGPT and products like Character.AI. The real starting guns for this AI wave in terms of a large number of founders jumping in was driven by two sets of launches. First were the launches of image-gen products like Midjourney and Stable Diffusion, followed a few months later by ChatGPT, which wowed the world, captured the public imagination, and was the AI startup big bang moment. ChatGPT truly highlighted the capabilities of these new forms of AI and the power of RLHF. OpenAI followed up with GPT-4, 4 months later. True enterprise adoption is still many quarters/years away. ChatGPT’s launch was the starting gun for mainstreaming that AI is a big deal in terms of new capabilities and kicked off the large scale enthusiasm, hype cycle, and adoption for generative AI. This launch was only 8-9 months ago, and GPT-4 did not come out until 5 months ago. Given that large enterprise planning cycles often take 3-6 months, and then prototyping and building will take a year for a large company, we are still very far away from peak AI usage or peak AI hype. Most large enterprises are still trying to analytically sort what “AI” means for them, and are still many quarters from embracing this new technology. 4 Waves of AI Adoption. Indeed, there are likely at least 4 waves of AI to consider in these early days. Wave 1: GenAI native companies . ChatGPT, Midjourney, Character.AI, Stable Diffusion, Github copilot, and other early launches that have now gained significant revenue and user traction. Obviously there are some great ML companies that pre-date GenAI that continue to participate in the current era (Hugging Face, Runway, Scale, WandB are a few that come to mind). Wave 2 (current wave): Early startup adopters and fast mid-market incumbents. This is the first wave of startups to launch on top of GPT-3.5/4 like Perplexity, Langchain, Harvey or others. In parallel, a small number of founder led multi-billion companies like Navan, Notion, Quora, Replit, and Zapier launched AI-powered products quickly and are the early adopters of the wave. Microsoft, Adobe, and Google are notable outliers as very large enterprise moving fast to AI - Microsoft likely due to its inside track with OpenAI, and Adobe as diffusion models tend to be cheaper and simpler to train than the large scale LLMs. Wave 3 (coming soon): Next wave of startups currently being founded. It will be exciting to see what is in this mix and may include new formats like voice and video in addition to using natural language in more verticals and more ways, as well as new types of infrastructure. Companies like Eleven Labs/ LMNT /LFG Labs, Braintrust , and many more will provide incremental experiences. There is a big wave of new startups coming. The current YC batch alone appears to have a 100 or more AI startups…. Wave 4 (coming 2024/2025?): First big enterprise adopters. Since enterprise planning and build cycles are so long, anticipate the first really products (versus demos or prototypes) from larger companies other than MSFT, Adobe, Google, Meta to start to show up in a year or two. This is when revenue to AI infra companies will start to ramp significantly relative to today, when hype will peak, and we will see further accelerated investment in AI. The Future Is Bright. There is enormous potential for this new wave of tech to impact humanity. For example, Google’s MedPaLM2 model outperforms human physicians to such a strong degree that having medical experts RLHF the model makes it worse (!). Given the strong potential it will be exciting to see all the immense innovation in education, healthcare, enterprise and consumer software, and other aspects of life coming via this tech breakthrough. We are only 8-9 months since chatGPT woke the world to this new era of AI, and exciting times are ahead as we follow this tech discontinuity down our timeline. It is the very earliest days of AI, and both peak hype and peak impact are still in the future. Lots more is still to come. MY BOOK You can  order the High Growth Handbook here . Or  read it online for free . OTHER POSTS. Firesides & Podcasts. Clem Delangue: Hugging Face, Open Source, AI. Dylan Field: Figma, AI & Design, Education. Reid Hoffman on AI, Big Tech, and Society. Sam Altman, CEO OpenAI. Emad Mostaque, Stability.AI. NoPriors AI Podcast. Markets: AI Safety: Technology vs Species Threats. Startup Decoupling and Reckoning. Defensibility and Competition. AI Platforms, Markets, and Open Source. Changing times (or, why is every layoff 10-15%?) AI Startup Vs Incumbent Value. AI Revolution - Transformers and Large Language Models. Startup Markets Summer 2022. False Narrative Around Theranos. Index Companies. Defense Tech. Collaborative Enterprise. Industry Towns: Where you start a company matters. Markets are 10X Bigger. Hot Markets 2019. End of Cycle? Machine Learning Startups. 3 Types Of Platform Companies. Defensibility and Lock-In: Uber and Lyft. Uber And Disruption. Who Cares If Its Been Tried Before? The Road To $5 Billion Is A Long One. How To Win As Second Mover. End Of Silicon Valley. Social Products. Hot Markets For 2015. Startup life. Startups Are An Act of Desperation. Back To The Office. Hiring Executives and Bad Advice. When executives break. Fear of Sales. A brief guide to startup pivots. 4 Ways Startups Fail. Founder Investors and Scout Programs. Better Meetings. Magic Startup Moments. Founder Investors & Scout Programs. Jobs, Wozniak, Cook. Co-Founders. How To Choose A Co-Founder. Unequal Cofounders. How To Fire A Co-Founder. Founders Should Divide and Conquer. Raising Money. Preemptive rounds. Don't Ask For Too Much Money. Building VC Relationships. Founders Should Divide And Conquer. Lead VC Vetos. What Is A Good VC? How To Choose The Right VC For You. Signs a VC Just Isn't That Into You. Series A Crunch. Questions VCs Will Ask You. How To Raise A Successful VC Round. Differences Between Funding Rounds: Series Seed, A, B, C... Financing Approaches Most Likely To Kill Your Company. Party Rounds: How to Get A High Valuation For Your Seed Startup. 20 Questions To Ask Yourself Before Raising Money. The 7 Types Of Angel Investors. Fundraising Will Take You 3 Months. How To Sell Secondary Stock. Old Crypto Stuff: Core Crypto Use Cases (2018) The Case For Ethereum. Bitcoin Network Effects. Cryptocurrency Incentives and Corporate Structures. Big Banks and Blockchains. Cryptocurrency's Netscape Moment. Thanks for reading Elad Blog! Subscribe for free to receive new posts and support my work. Subscribe. 52. Share. Previous.