AI Chip Startup Etched Completes Tape-Out, Raises $800 Million in Financing

A company that hasn't yet mass-shipped chips is attempting to rewrite the cost-performance formula for AI computing power.

Recently, Etched, an AI chip company founded by Harvard University dropouts, secured a new round of financing nearing $500 million. The startup's valuation is reportedly $5 billion, with total funding approaching $1 billion.

This round was led by Stripes, with participation from Peter Thiel, Positive Sum, and Ribbit Capital. Previous supporters include Primary Venture Partners, as well as notable angel investors such as former GitHub CEO Thomas Dohmke and former Coinbase executive Balaji Srinivasan.

Interestingly, the company neither makes general-purpose GPUs nor aims to completely replace NVIDIA. Instead, it seeks to perfect one thing: making Transformers run cheaper.

Fig. Sohu Chip (Source: Etched)

From the GPU market perspective, NVIDIA dominates. Market predictions suggest NVIDIA's cumulative datacenter sales will exceed $500 billion by the end of 2026. Etched's assessment of the market is that over the past few years, compute density (TFLOPS/mm²) has only improved by about 15%. New generation GPUs (NVIDIA B200, AMD MI300X, Intel Gaudi 3, AWS Trainium2, etc.) now treat two chips as one card, effectively "doubling" performance.

Now, with large model training slowing down and inference exploding, models severely underutilize compute and search capabilities regarding inference time. The demand for computing power is no longer just about FLOPS, but a comprehensive competition on cost, latency, and energy consumption. Leveraging better algorithms and faster hardware is expected to improve this situation.

Etched builds custom ASICs designed for AI models with Transformer architecture. Etched claims this chip, named Sohu, is the "fastest AI chip in history." Under specific test configurations, Sohu achieves a throughput of over 500,000 tokens per second when running the Llama 70B model, enabling products that cannot be built relying solely on GPUs.

"When running text, image, and video Transformers, Sohu is one order of magnitude faster than NVIDIA's Blackwell GB200 GPU and costs less," Gavin Uberti, Co-founder and CEO of Etched, told media outlets. "Under specific inference configurations given by Etched, a server composed of 8 Sohu chips can replace 160 H100 GPUs. For enterprises requiring specialized chips, Sohu will be a more economic, efficient, and eco-friendly choice."

(Source: Etched)

Although this poses a challenge to NVIDIA's technology, the company's goal is not to fully replace NVIDIA, but to "bypass" its technical roadmap. Unlike general-purpose GPUs, Sohu adopts a highly specialized strategy: aiming to lower energy consumption while running Transformer models more efficiently than general GPUs.

Today, AI model training costs exceed $1 billion, while their inference application scale may exceed $10 billion. At such a massive scale, a 1% performance improvement is enough to support a custom chip project costing $50 million to $100 million.

According to Etched's official website, this chip is used for production-level inference, improving compute efficiency per dollar (and watt) by one order of magnitude in scenarios such as dense models, sparse modalities, and diffusion. If successful, this means driving AI hardware development beyond just pushing scale.

Fig. Hardware performance comparison running Meta's open-source model Llama 70B (Source: Etched)

Public information shows Sohu is manufactured using TSMC's 4nm process, and secures HBM memory and server hardware supply from upstream suppliers to support chip and server integration production capabilities.

In reality, many startups and tech giants are developing chips specifically for running AI models, known as inference chips. Examples include Meta's MTIA, Amazon's Graviton and Inferentia, etc.

However, the uniqueness of Etched's chip lies in "subtraction": it focuses solely on running Transformer models and cannot run other AI models including CNN, LSTM, and SSM. Precisely because of this, it avoids hardware components unrelated to this and software overhead brought by chips for other types of workloads.

Additionally, Etched partnered with Decart to launch the AI-generated game Oasis, which can be understood as an unofficial "Minecraft." Game visuals are synthesized in real-time by generative models during player interaction, rather than traditional pre-made asset rendering. Etched claims this model runs more than 10 times faster on Sohu.

In 2022, Gavin Uberti and Chris Zhu dropped out of Harvard University to found Etched, headquartered in San Jose, California, USA. Later, Co-founder and President Robert Wachen and CFO Mark Ross joined them.

Fig. From left to right: Robert Wachen, Gavin Uberti, and Chris Zhu (Archive Image)

Company Co-founder and CEO Gavin Uberti previously served as a mathematics researcher and AI compiler expert at Harvard University, writing the Cortex-M backend for Apache TVM. Co-founder Chris Zhu previously served as a Harvard mathematics and high-performance computing researcher, a Thiel Fellow, and published research in the field of combinatorial mathematics.

The two are Harvard alumni who studied mathematics and computer science respectively. At the end of 2022, while conducting research, they realized that performance per dollar could reach 140 times that of traditional graphics processors when running generative AI models. Seeing a very short window of engineering opportunity, they decided to use this discovery to do something bigger—dropping out of Harvard to start a company.

Co-founder and President Robert Wachen previously served as Co-founder of startup incubator Prod, whose incubated enterprises have a valuation exceeding $50 billion. CTO Mark Ross previously served as CTO of Cypress (the company was acquired for $9 billion).

Gavin Uberti and Chris Zhu previously stated in media interviews that although they currently only produce chips capable of running AI language models for generative text, the entire company should not rely solely on a single product. The company has many new technologies, including image and video generation and protein folding simulation. Etched's longer-term vision is to manufacture other chips for different types of AI models.

Fig. Etched Core Members (Source: Etched)

Currently, there are also related companies demonstrating ideas similar to Etched. For example, AI chip startup Perceive showcased a processor Ergo 2 with Transformer hardware acceleration capabilities, capable of edge inference for Transformer models with over 100 million parameters, processing video at higher frame rates, and inferring multiple large neural networks simultaneously.

Smart chip company Groq's LPU (Language Processing Unit) is an AI ASIC tailored for inference scenarios, aiming to run large models (Transformer inference) with high efficiency and low latency.

Additionally, AI chip company Tenstorrent develops highly scalable AI processors (such as Grayskull/Wormhole/Blackhole) based on RISC-V architecture, and is actively seeking alternatives to GPUs for AI inference or training.

Despite the promising prospects, it must be seen that Etched's success is betting on Transformer models continuing to play the role of mainstream architecture. If this assumption changes in the future, everything may need to start over. It can be said that Etched is not betting on a single chip, but on an era judgment: the future of AI may not need so much "all-powerful," but just "just right" is enough.

References:

https://techfundingnews.com/nvidia-rival-ai-chip-maker-etched-founded-by-harvard-dropouts-lands-500m-at-5b-valuation/

https://www.canopy.space/members/member-profile-etched/

https://www.embedded.com/ai-chip-features-hardware-support-for-transformer-models/

https://techcrunch.com/2024/06/25/etched-is-building-an-ai-chip-that-only-runs-transformer-models/

https://siliconangle.com/2024/06/25/transformer-model-chipmaker-etched-ai-raises-120m-challenge-nvidias-market-dominance/

https://www.cnbc.com/2024/06/25/etched-raises-120-million-to-build-chip-to-take-on-nvidia-in-ai.html

Operations/Layout: He Chenlong

Source: DeepTech

Disclaimer: This article is for information exchange only and does not constitute any investment advice. Stock market involves risks, investment requires caution. Return to Sohu to view more.

This page provides an editorial summary based on publicly available information. It is not a republished article. Use the source link below for the original report.