Who Are the Top Players in the Red-Hot AI Chip Market?

Daniel Morgan, Senior Portfolio Manager

Nvidia (NVDA) continues to be in the pole position in the artificial intelligence (AI) race, but what other semiconductor companies are in the pack? Advanced Micro Devices (AMD), Broadcom (AVGO), Intel (INTC) and Marvell Technology (MRVL)? What about other data center/search companies like Amazon, Microsoft, Alphabet and Meta? That not only buy AI chips to run their sophisticated cloud/internet systems from the likes of NVDA, but are in the process of developing their own proprietary AI chips?

NVDA’s management does not seem overly concerned about competition given its strong performance lead. NVDA today accounts for more than 70% of AI semiconductors sales with Google, Meta, Amazon and Microsoft, all producing their own AI chips. NVDA continues to dominate performance benchmarks and the company’s eco-system advantage would be hard to match. Given the strong demand for its Data Center products, NVDA has extended visibility for the next several quarters and has secured more supply to support significant revenue growth in the second half of 2024. Next-generation products are an important driver of order activity, lead times and competition. NVDA recently announced a new AI GPU, the B100 (Blackwell), which will be a substantial upgrade from H100 (Hopper). The B100 is system compatible and will be priced more aggressively than initial expectations. This renders the lead-time analysis as somewhat irrelevant, as the demand should transition rapidly from H100 to B100, as production ramps up. NVDA is quite motivated to use B100 to blunt the momentum of competition from AMD/INTC/MRVL.

Advanced Micro Device’s recent guidance for AI chip sales in FY2024 is what has excited investors! AMD is one of the few chipmakers capable of making the kind of high-end graphics processing units (GPUs) needed to train and deploy generative AI models. AMD has a host of new AI chips to be released – MI300A and MI300X – that are “on track” for volume production in the current quarter. During a conference call, AMD CEO Lisa Su said, “We now expect data center GPU revenue to exceed two-billion dollars in 2024 as revenue ramps throughout the year.” AMD may become a formable competitor to market leader NVDA. Further, AMD is working directly with Microsoft to develop additional AI chips under the code name Athena.

Broadcom’s AI opportunity continues to expand as the current pipeline is on track to generate $8.0-$9.0 billion in revenues for FY2024, (based on an estimate for company revenues for FY2024 of $51.94 billion), representing 15%- 17% of the total business. Most of the AI revenues are generated by AVGO’s custom AI ASIC programs. Joint ventures to produce AI chips include Alphabet’s TPU AI ASIC program and the start of the ramp-up for Meta’s 3nm AI ASIC program later this year. AVGO has co-designed every generation of Alphabet’s TPU (Tensor Processor Unit) so far and is currently designing with GOOGL the next gen v6 TPU at 3nm, which is expected to reach production later this year.

Intel’s management views the current “AI Craze” as a positive-growth catalyst for the semiconductor industry – even though GPUs like those produced by Nvidia, not CPUs like the ones Intel focuses on, are getting all the attention. In response to this demand, Intel is producing two AI chip initiatives – Falcon Shores and Gaudi2/Gaudi3. The next-gen Falcon Shores discrete GPU will focus on HPC and AI-optimized compute with the planned integration of AI processors from the company’s Habana Gaudi (Gaudi2/Gaudi3) family. The Falcon Shores GPUs will be the successor to the company’s current Max Series GPUs (Ponte Vecchio) and is now scheduled to be released in the 2025 timeframe. Intel’s forthcoming "Falcon Shores" chip will have 288 gigabytes of memory and support 8-bit floating point computation. Those technical specifications are important as AI models similar to services like ChatGPT have exploded in size, and businesses are looking for more powerful chips to run them.

Marvell Technology is expected to be major beneficiary of the aggressive spending on generative AI by its cloud customers and is on track to double its AI-based revenue to $400 million in FY24, driven by strong demand pull for Marvell’s 800G PAM4 DSP chipsets and the 400ZR DCI solutions. Marvell recently announced a 5nm Transmit-Only 800G PAM4 Optical DSP for AI and Cloud Interconnects, code named Spica Gen2-T. Industry-wide shipments of PAM4 optical DSPs are expected to more than triple by 2029 due to the urgent demand for bandwidth. At the same time, the growing pervasiveness and importance of optical technology in data center architectures will put a premium on efficient solutions.

Amazon has recently jumped into the “homegrown AI chip” pond with Alphabet and Meta, with the recent release of two new chips called Graviton4 and Tranium2. These CPUs will accelerate the speeds of Amazon’s flagship AWS data center service. Amazon recently released two AWS-designed chip families, delivering advancements in price performance and energy efficiency for a broad range of customer workloads, including machine learning (ML) training and generative artificial intelligence (AI) applications. Graviton4 and Trainium2 mark the latest innovations in chip design from AWS. With each successive generation of chip, AWS delivers better price performance and energy efficiency, giving customers even more options — in addition to chip/instance combinations featuring the latest chips from third parties like AMD, Intel and NVIDIA — to run virtually any application or workload on Amazon Elastic Compute Cloud (Amazon EC2).

Microsoft is finally making custom chips — and it is all about AI. The Azure Maia 100 and Cobalt 100 chips are the first two custom silicon chips designed by Microsoft for its cloud infrastructure. Microsoft’s Azure Maia AI chip and Arm-powered Azure Cobalt CPU are arriving in 2024, on the back of a surge in demand this year for Nvidia’s B100 GPUs, which are widely used to train and operate generative image tools and large language models. The new Azure Maia AI chip and Azure Cobalt CPU are both built in-house at Microsoft, combined with a deep overhaul of its entire cloud server stack to optimize performance, power and cost.

Alphabet has been working on its own custom AI chips for years. The company's Tensor Processing Unit, or TPU, was first announced in 2016. A GPU is more general purpose, making it useful for a wide variety of workloads in addition to AI, but potentially less efficient. GOOGL is in the process of creating the next gen v6 TPU at 3nm that is expected to reach production later this year. GOOGL may some day rely on the Tensor Processing Unit to power Google Cloud. Google Cloud is not the first major cloud provider to launch virtual machines powered by Nvidia's H100 – Amazon Web Services announced a similar product in July, and Microsoft Azure did the same earlier this month. However, the company is betting that cloud customers want options. Its efficient and cost-effective TPU-powered services could give it an edge as cloud providers scramble to win AI workloads.

Meta has dived headfirst into the AI silicon race with the introduction of MTIA v1. In the ever-evolving AI landscape, Meta has recently introduced its new AI chip, “Meta Training and Inference Accelerator,” or simply called MTIA. The MTIA chip shows marginal improvements in efficiency for simple low- and medium-complexity inference applications, while it currently lags behind GPUs for complex tasks. However, Meta is planning to match GPU performance through software optimization later down the line. MTIA represents a sharp turn in the road for Meta, as the company redirects resources toward AI, away from the money-losing Metaverse initiative. Many Investors are betting that maybe AI can help revive growth. Building AI infrastructure feels, in the current macroeconomic environment, like the right choice for Meta.

With so many non-traditional semiconductor companies now producing AI chips, which one will come out as the biggest winner and lead the pack? A traditional semiconductor company like NVIDIA, AMD or an upstart, like online marketplace/social media giants Amazon or Meta? Gartner Inc. recently estimated that semiconductors designed to execute AI workloads represented a $53.4 billion revenue opportunity for the semiconductor industry in 2023, a 20.9% increase from 2022. AI semiconductor revenue will continue to experience double-digit growth through the forecast period, increasing 25.6% in 2024 to $67.1 billion. By 2027, AI chip revenue is expected to be more than double the size of the market in 2023, reaching $119.4 billion. So there seems to be plenty of AI chip demand around to lift the entire sector!

My hunch is that the traditional chip makers – NVIDIA, AMD, AVGO and INTC – would have a leg up on the data center/search companies – Amazon, Microsoft, Alphabet and Meta, for example – in regard to innovation and technology and based on the many years of experience in semiconductor development as a core company focus. With that said, though, I would expect data center/search companies (Amazon, Microsoft, Alphabet and Meta) – that today rely heavily on AI chips produced by traditional chipmakers – will begin to use their own proprietary chips going forward for AI uses and become less reliant on traditional providers. It’s similar to what Apple has done over the years with its a16 Bionic chip that provides raw processing power to an iPhone, as opposed to using a DSP chip from one of its many U.S. vendors like Broadcom, Qualcomm, Intel, On and Texas Instruments. Amazon, Microsoft, Alphabet and Meta will most likely only produce AI chips for their own networks and software, not looking to market the chips to other end users. With the AI chip space being the fastest growth segment within the industry, which includes traditional sectors like PCs, servers, smart phones, automotive and industrial, I expect the AI tide to rise – benefiting all participants!

How Does the Competition Stack Up for AI Chips?

GPU = Graphics Process Unit; CPU = Central Processor Unit; TPU = Tensor Processor Unit; DSP = Digital Signal Processor; ASIC = Application Specific Integrated Circuit; APU = Accelerated Processing Unit; * = China Market version/configuration