AI chips

With all the buzz surrounding AI and the Microsoft ChatGPT deal, I thought it might be interesting take a look at the overall health of the semiconductor industry and conduct a brief overview of some of the top semiconductor companies that are offering AI chips.

The near-term fundamentals of the global semiconductor market point to revenues projected to decline 11.2% in 2023, according to the latest forecast from Gartner, Inc. In 2022, the market totaled $599.6 billion, which was marginal growth of 0.2% from 2021. The short-term outlook for the semiconductor market has deteriorated further. As economic headwinds persist, weak end-market electronics demand is spreading from consumers to businesses, creating an uncertain investment environment. In addition, an oversupply of chips, which is elevating inventories and reducing chip prices, is accelerating the decline of the semiconductor market this year.

That said, the PC, tablet and smartphone semiconductor markets are stagnating. The combined markets will represent 31% of semiconductor revenue in 2023 and total $167.6 billion. These high-volume markets have saturated and become replacement markets devoid of compelling technology innovation. In contrast, both the automotive and industrial, as well as military/civil aerospace, semiconductor markets will achieve some growth. For example, the automotive semiconductor market is forecast to grow 13.8%, reaching $76.9 billion in 2023.

This outlook from Semiconductor Industry Association (SIA) is echoed by Gartner’s most recent global Wafer Fabrication Equipment (WFE ) semiconductor equipment spending forecast (April 2023) that calls for a 20% year-over-year (YoY) decline in FY2023 to revenues of $80.5 billion, with just a modest improvement of 0.6% in FY2024. This weakness has been caused by elevated back logs and delays in equipment deliveries given capex cuts from major chip manufacturing fabs, like TSM and Samsung. The slowdown in equipment used to produce memory chips (DRAM and NAND), has been greatly impacted.

In terms of the underlying technology and the manufacturers behind them, the AI gold rush has attracted interest from every corner of the technology world. This has ranged from graphics processing unit (GPU) and central processing unit (CPU) companies to Field Programmable Gate Arrays (FPGAs) firms, custom application-specific integrated (ASIC) markers and more. There is a need for inference at the edge, inference at the cloud, training in the cloud – AI processing at every level, served by a variety of processors. But among all of these AI facets, the most lucrative market of all is the one at the top of this hierarchy: the datacenter. Expansive, expensive, and still growing by leaps and bounds, the datacenter market is the ultimate feast or famine setup, as operators are looking to buy nothing short of massive quantities of discrete processors.

So based on this not-so-rosy semiconductor industry outlook for 2023, how can the Philadelphia Semiconductor Index (SOX) be up 38% YTD! The answer is: all the investor enthusiasm surrounding AI chip makers. So, let’s take brief look at some of the top AI chip manufacturers and see how their technology stacks up!!

Advanced Micro Devices (AMD)

While AMD’s EPYC server CPU momentum is to remain the primary focus/growth driver through 2023, as one looks into 2024 and beyond, AMD’s ability to participate in the broader (AI-driven) datacenter silicon market will likely become an increasingly important part of a positive investment thesis. The Datacenter GPU market is expected to grow from $10.3 billion in 2022 to $33.8 billion by 2027, a +27% compound annual growth rate (CAGR), a market in which AMD's current revenue share of the datacenter space is estimated at sub-3% range.

AMD has recently unveiled its AI chip blueprint that includes: 1) The AMD Alveo™ V70 AI Accelerator with industry-leading performance and energy efficiency for multiple AI inference workloads; 2) The world’s first integrated datacenter CPU and GPU, the AMD Instinct™ MI300 accelerator. These accelerators complicate AMD’s core AI chip offering the MI300. 3) The MI300 is a family of large processors, and it is modular. AMD can utilize either CDNA3 GPU IP blocks or AMD Zen 4 CPU IP blocks along with high-bandwidth memory (HBM.) The MI300X is a GPU-only version is designed to go toe to toe with NVIDIA’s H100 Hopper AI accelerator and is optimized for large language models and generative AI. MI300X will have a total of 192GB of HBM3 memory on board delivering 5.2 TB/s of bandwidth and is comprised of 153 billion transistors across a 12 chiplet design. Expect the MI300 adoption at some hyperscale cloud customers could materialize into late 2023/2024.

Amazon.com is considering the use of AMD’s (AMD +2.3%) new AI chips for its cloud business. Looking ahead, Microsoft is providing engineering resources to support AMD’s developments as the two companies join forces to compete against Nvidia, which controls an estimated 80% market share in the AI processor market. In turn, AMD is helping Microsoft to develop its own in-house AI chips, code-named Athena. On another front, the AMD + Xilinx Integration should also begin to bear fruit. As we look into 2024 expect to hear more about the integration of AMD + Xilinx for 2.5D / 3D heterogeneous / hybrid compute architectures, including AMD's integration of XDNA AI Engines w/in the EPYC server CPU roadmap. The company's MI200 product cycle was highlighted as being positioned for HPC-first, while the MI250X further solidified AMD's HPC software ecosystem buildout and provided the foundation for the company to emphasize/build out its software ecosystem for AI. AMD's MI300 APUs (DC GPU + CPU) reflects the company's push to participate in the full breadth of the GPU market.

Alphabet

Alphabet’s AI chip technology centers on its Tensor Flow unit. TensorFlow was developed and first released by Google in 2015. Google’s latest release, the TPU v4 chip, was built to power AI-related workloads and services offered in Google Cloud. The TPU v4 chip has boosted the performance of its TPU hardware by more than two times over the previous TPU v3 chips, bringing critical new power and promise to machine learning (ML) training speeds on the Google Cloud Platform. The TPU v4 is connected together into supercomputers, called pods. A single v4 pod contains 4,096 v4 chips, and each pod has 10 times the interconnect bandwidth per chip at scale, compared to any other networking technology. TPUs are Google’s custom-developed application-specific integrated circuits (ASICs), which are used to accelerate ML workloads. Developers can use Google Cloud TPUs and Google’s TensorFlow open source ML software library to run their ML workloads. Google Cloud TPU is designed to help researchers, developers and businesses build TensorFlow compute clusters that can use CPUs, GPUs and TPUs as needed. TensorFlow APIs allow users to run replicated models on Cloud TPU hardware, while TensorFlow applications can access TPU nodes from containers, instances or services on Google Cloud.

Broadcom

Broadcom is riding high after signing a new multiyear, multibillion-dollar deal with Apple to provide 5G radio frequency components for iPhone devices, extending a previous agreement from 2020. The collaboration will include wireless connectivity parts and FBAR filters, which help mobile phones focus airwave signals and reduce interference. Broadcom’s chips do a lot of support work for AI services. AVGO’s switch chips control the flow of information between computers and are seen as a crucial part of the infrastructure needed to expand the technology. However, it would be overly optimistic to think that AVGO will show a major boost of adrenaline (like NVDA did on its 2Q24 earnings release) from AI in the near term. AVGO is seeing a pickup in demand from its hyperscale customers for network chips deployed in AI; management recently estimated that ethernet switches deployed in AI accounted for $200 million in sales for 2022.

In 2023, the company expects this number to grow to well above $800 million. Management clarified that these were related to the current generation of products and not dependent upon future products such as Jericho3-AI, which will ship in 2024. Jericho3-AI is designed to connect supercomputers and features a high-performance fabric for AI environments. Jericho3-AI is targeted at AI and ML backend networks where the switch fabric handles spraying of traffic on all network links and reordering of that traffic before delivering to the endpoints. New products will only expand these revenue opportunities as AVGO can command higher ASPs with their leading technologies. During AVGO’s 2Q23 results, management called out its Networking revenue related to AI infrastructure, which is expected to grow from 10% of Semi revenue in FY22 to 15% in FY23 and 25% in FY24. This implies growth of +60-70%, plus over the next couple of years driven by Gen AI infrastructure buildouts by cloud providers. Analysts speculate that AVGO helps design proprietary AI chips for GOOGL and META, among others. Networking represents 39% of the Semiconductor segment and was guided to increase by +20% YoY, maintaining its current growth trajectory.

Intel

Intel’s management views the current “AI Craze” as a positive growth catalyst for the semiconductor industry, even though GPUs like those produced by Nvidia — not CPUs like the kind Intel focuses on — are getting all the attention. Intel recently announced that the company is moving away from plans to provide an integrated CPU + GPU (XPU) architecture in the forthcoming Falcon Shores chip to now focusing on only providing a discrete GPU – still code-named Falcon Shores. Intel notes that this change reflects the fact that the company now sees a more dynamic computing environment than even just a year ago. The next-gen Falcon Shores discrete GPU will focus on HPC and AI-optimized compute with the planned integration of AI processors from the company’s Habana Gaudi family. The Falcon Shores GPUs will be the successor to the company’s current Max Series GPUs (Ponte Vecchio) and is now scheduled to be released in the 2025 timeframe. Intel’s forthcoming "Falcon Shores" chip will have 288 gigabytes of memory and support 8-bit floating point computation.

Those technical specifications are important as AI models similar to services like ChatGPT have exploded in size, and businesses are looking for more powerful chips to run them. The details are also among the first to trickle out as Intel carries out a strategy shift to catch up to Nvidia, which leads the market in chips for AI, and AMD, which is expected to challenge Nvidia's position with a chip called the MI300. Intel, by contrast, has essentially no market share after its would-be Nvidia competitor, a chip called Ponte Vecchio, suffered years of delays.

Marvell Technologies

Marvell reported 1Q24 results above consensus on better-than-expected Data Center (driven by strong growth in AI-based optical revenues) and provided a slight improvement for the 2Q24 growth outlook, which reflected quarter-over-quarter (QoQ) revenue stabilization (up 1% QoQ). This is expected to be driven by strength in 5G wireless, auto, and AI-based revenues from the 800G PAM4 optical for NVIDIA/Google AI clusters and 400ZR datacenter interconnect solutions. Unfortunately, this was offset by continued soft demand in certain parts of its business — enterprise networking, wired carrier demand, and datacenter on-premises/legacy, resulting in muted overall growth.

MRVL is expected to be major beneficiary of the aggressive spending on generative AI by its cloud customers and is on track to double its AI-based revenue to $400 Mn in FY24 driven by strong demand pull for Marvell’s 800G PAM4 DSP chipsets and the 400ZR DCI solutions. Over the long haul, MRVL now estimates overall cloud ASIC revenue to be higher than its prior forecast of $800 million driven by compute AI ASICs. The overall strength surrounding the 800G PAM4 AI chip is being offset by the general deacceleration in CAPEX growth for large datacenters.

Nvidia

Nvidia currently dominates the market for AI chips with more than 80% of the market share. NVDA remains the king with its DGX H100 (Hopper) and A100 for AI, but challenges are ahead with competition and restrictions. Overall, NVDA still reigns supreme in the world of AI and accelerated computing with breadth and depth of offerings from hardware to a full software training/AI stack. The company announced its DGX H100 in March, built on its Hopper platform (TSMC 4nm, HBM3 memory, 80B transistors). The DGX H100 provides 32 PFLOPs ("petaflops") of AI performance at FP8, 6 times versus the A100, and is targeting increasing AI challenges as transformer models expand toward over one trillion parameters. Expect the DGX H100 is set for volume ramp in 2023, but could see some challenges to Chinese demand, given recently announced bans for both the H100 and prior-gen A100. Recently, Nvidia reported a blowout 1Q24 result with revenue of $7.192 billion (up 19% QoQ and down 13% YoY) that handily beat Street expectations of $6.516 billion.

The big news came in the Gaming segment that was up 22.3% QoQ $2.24 billion, while Data Center revenue of $4.284 billion was up 18.5% QoQ. The icing on the cake was NVIDIA’s guidance for the upcoming 2Q24 projecting revenues of $11 billion at the midpoint (up 52.9% QoQ and 64.1% YoY) that came in well ahead of the Street estimate of $7.146 billion, with Data Center clearly driving the lion's share of the growth. It has been estimated that approximately that 65% of NVDA’s overall datacenter revenues are driven by generative AI with 90% of that coming from deep learning and 10% from inference. This marks the first true evidence in the semiconductor sector that shows that the increased demand for AI chips, especially in the datacenter space, is actually providing a significant catalyst. The future looks bright for NVDA, as the company recently announced a new computer system, software and services-generative AI that includes the DGX GH200 for the next-gen models powering AI. This new Grace Hopper chip will be targeted toward developing and supporting large language models.

Qualcomm

Investors know QCOM’s proprietary CDMA Technology and as a top maker of modems chipsets that are the workhorse of a smart-phone/iPhone. But, among all AI’s facets, the most lucrative market at the top of this hierarchy is the datacenter. Expansive, expensive, and still growing by leaps and bounds, the datacenter market is the ultimate feast or famine setup, as operators are looking to buy nothing short of massive quantities of discrete processors. Qualcomm, one of the last juggernauts to sit on the sidelines of the datacenter AI market, is finally making its move. Qualcomm’s AI efforts appear to focus on a future whereby AI will require more computing power than what the cloud can provide. As growth in the number of connected devices and data traffic swells datacenter costs climb, and it will no longer be feasible to send all that data to the cloud.

The Qualcomm Cloud AI 100, designed for AI inference acceleration (not training), addresses unique requirements in the cloud, including power efficiency, scale, process node advancements, and signal processing — facilitating the ability of datacenters to run inference on the edge cloud faster and more efficiently. Qualcomm Cloud AI 100 is designed to be a leading solution for datacenters that increasingly rely on infrastructure at the edge-cloud. As AI comes to the edge at scale with large language models, Qualcomm sees an opportunity for replacement rates to increase. Low power AI engine on a chip is a key differentiating technology, creating incremental monetization opportunity. As large language models (LLMs) continue to spread, Qualcomm plans to expand the capacity of its low power AI engine and apply it to LLMs, which will be key to running LLMs on devices at the edge while creating a key technological advantage for smartphone, PC, automotive and industrial edge devices.

CFRA Research Analyst Angelo Zino has dubbed the “Four Horseman of AI” as Advanced Microdevices, Broadcom, Marvell Technology and Nvidia. This is a takeoff on the “Four Horseman” of the Dom.Com era of “Cisco, Dell, Intel & Microsoft.” Four Horseman is rooted in the nickname given the Notre Dame football backfield of 1924.

Nvidia’s top pole position with 80% MS in AI/accelerate compute and as the one-stop solution provider with its portfolio of silicon, software/managed cloud services, hardware systems, and full-stack ecosystem for training/deploying AI models gives them a leg over the competition. Investor’s recent enthusiasm for AI has driven NVDA’s shares up +268% YTD! NVDA’s valuation is not for the faint-hearted as the stock is trading at 54x FY2024 EPS Estimates of $6.58 per share!

So, which chip makers are the “Best of the Rest” and could duplicate Nvidia’s expected acceleration in growth from AI datacenter chips? As we discussed earlier, AMD has The MI300X a GPU-only version is designed to go toe to toe with NVIDIA’s H100 Hopper AI accelerator, and is optimized for large language models and generative AI. Further, AMD is working directly with Microsoft to develop additional AI chips under the code name Athena. Broadcom’s Jericho3-AI will not be ready to ship until 2024. Jericho3-AI, is designed to connect supercomputers and features a high-performance fabric for AI environments. MRVL is expected to be major beneficiary of the aggressive spending on generative AI by its cloud customers and is on track to double its AI-based revenue to $400 million in FY24, driven by strong demand pull for Marvell’s 800G PAM4 DSP chipsets and the 400ZR DCI solutions. Trying to predict which one of the chip players will be able to break out of the pack and make it to Place (No. 2) or Show (No. 3) position behind NVDA is still difficult to foresee.

Daniel Morgan Senior Portfolio Manager