Semi Doped

Vik Sekar and Austin Lyons tackle the biggest bottleneck in inference: moving data. They break down the three tiers of datacenter networking — scale up, scale out, scale across — and the core engineering trade-off at each layer: copper vs. optics. Topics include Nvidia's extreme measures to keep scale-up fabric electrical (a 78-layer mid-plane PCB), why Co-Packaged Optics is the "holy grail" everyone wants and no one can ship, and the serviceability problem standing in its way.

Key Takeaways:

Al Yuen, CEO of PicoJool, talks with Austin about using VCSELs for scale-up optical interconnects in AI data centers. Al explains why gallium arsenide (GaAs) supply is unconstrained while indium phosphide (InP) is limited, and how PicoJool can leverage existing supply chains to ship in the millions per month. They cover the roadmap from 1.6T to 12.8T, the trade-offs among 8×200G, 16×100G, and 32×50G, and the push to train the next generation of photonics engineers.

Vik welcomes Val Bercovici from Weka to discuss the rapidly evolving landscape of AI memory and storage. Val explains how Weka's architecture leverages high-bandwidth networks to make storage faster than motherboard DRAM. They dive into KV cache optimizations, the future of NAND flash tiers, and the role of CXL in AI inference. The episode concludes with a look at predictive memory offloading and the AI flywheel.

Chapters:
0:00 Welcome Val Bercovici, Weka
1:59 Memory situation and model routing
3:50 KV cache offloading to CMX
6:10 Network faster than motherboard
13:10 Weka as AI memory infrastructure
14:45 Inference...

Austin and Vik break down the current memory crisis, where AI demand is driving unprecedented profits for memory makers like Micron, while simultaneously causing inflation and price hikes for consumer electronics. They discuss how even premium brands like Apple are struggling to secure supply, leading them to controversial suppliers. The hosts also explore the long-term implications of AI's insatiable memory appetite and its impact on both data center and consumer markets.

Chapters:
0:00 Memory crisis hits
1:20 AI impacting consumers
3:00 AI causing inflation
6:48 Consumer demand drop?
8:49 AI demand inelastic
10:56 Long-term memory outlook
11:02...

Qualcomm flew Austin to New York for its investor day, where the communications company laid out a plan to make data center, automotive, and IoT two-thirds of its business by FY29. Austin was in the room and asked Cristiano Amon a question on the record. Vik watched the whole thing on YouTube. Together they break down what actually changed.

The technical centerpiece is High Bandwidth Compute (HBC): stacking LPDDR on top of logic to expose the whole face of the chip for interconnects, claiming up to 100x more lanes and a path around the HBM bandwidth bottleneck...

New episode: Advanced packaging for AI chips, from wire bonds to TSMC CoWOS and Intel EMIB.

Packaging is no longer an afterthought. It is the chip, and Intel's EMIB challenges TSMC's CoWOS.

This episode is brought to you by SambaNova. Try SambaNova's fast inference today at the SambaNova Dashboard!

...

Austin and Vik discuss their recent experience at Computex, where they met for the first time in person after six months of podcasting together.

They share insights about the massive show, the people they connected with, and the exciting developments in AI hardware and interconnect technology.

Connect with Vik and Austin via a daily free newsletter:
https://www.semidoped.com

Vik's Paid Substack: https://www.viksnewsletter.com
Austin's Paid Substack: https://www.chipstrat.com

Chapters

00:00 Meeting in Person for the First Time
03:05 Experiencing Computex...

Huawei dropped a paper claiming 1.4nm-class performance without EUV, and the internet immediately declared ASML dead and US export controls useless. Austin and Vik recorded one day after Memorial Day to unpack what Huawei actually announced at ISCAS 2026 — and why the "EUV killer" headline gets the story backwards.

They walk through the tau scaling law (tau is delay, and the idea is to attack it at the system level instead of the transistor), logic folding via hybrid bonding, the Kirin 2026 that doubles transistor count without shrinking, and who can actually manufacture stacked logic. Then the other tau kn...

Spend one hour here and you've caught up on the entire arc of semiconductor lithography. Austin and Vik run a masterclass on the technology that decides who gets to make leading-edge chips, and why so few companies can afford to.

The thread is economics. An EUV machine runs about $400 million, a new fab needs roughly 15 of them, and the total bill clears $20-30 billion before a single wafer ships. Austin and Vik trace the whole story: Rock's Law and the cost of a fab, what it actually takes to build one, the evolution from 193nm DUV through...

Cerebras IPO is the only thing to talk about this week. 🔥

IPO prices at $185/share. Pops nearly 70% right after. The first wafer-scale chip company to make it public — after a 40-year curse killed every prior attempt.

A water-cooler-style convo on what Cerebras actually builds, why a 23 kW wafer is a power and cooling nightmare, why 44 GB of SRAM is both the magic and the wall for LLM inference, and the cursed Trilogy Systems saga that Gene Amdahl tried — and failed — to pull off in 1983.

Why does Cerebras leave the whole wafer intact instead of dicing...

Gimlet Labs runs an inference cloud built on heterogeneous silicon. Their software traces a PyTorch workload, segments it into its component parts, and schedules each piece onto the best-suited hardware — connecting chips from different vendors on a single high-speed fabric.

In this interview, Gimlet co-founder Natalie Serrino and former Intel executive Beltir walk through the architecture (graph trace, optimal split points, lowering each segment to TensorRT on NVIDIA and equivalents elsewhere), the three customer segments they sell into (frontier labs, sovereign clouds, AI natives), and a concrete demo: on GPT-OSS 120B at 8K input / 1K output, running th...

What's common to optics and power that ruins everything in the era of AI?

Resistance.

The same physics that drove interconnects to optics is now driving low-voltage power delivery up to 800V. Austin Lyons (Chipstrat) and Vik Sekar (Vik's Newsletter) unpack it using the Kyber rack as an example.

At 600kW and 48V, you're pushing 12,500 amps through a single rack. Power loss scales with I². The math doesn't work. The fix is 800V — and the parts come straight from the EV traction inverter ecosystem (SiC, GaN, IGBTs).

We cover the full gri...

The hyperscaler memory tax quarter.

More CapEx? Pssh. We knew flops needed scaling.

But $25B at Microsoft alone just to pay higher component prices? 

A memory tax. That's the news.

NAND? Sold out. HBM? Sold out.

What we cover:

New S...

Google's Cloud Next 2026 keynote? Fire. 🔥

The TPU is now two chips instead of one — 8t for training, 8i for inference — but more interestingly, it's two scale-up networking topologies too.

Austin Lyons (Chipstrat) and Vik Sekar (Vik's Newsletter) walk through what actually changed, one day after the announcement. OCS? Yes. AECs? Yep. Copper? Yep. Optics? Yep.

We cover Virgo (Google's 47 petabit/second scale-out fabric, built entirely on OCS), Boardfly (the new scale-up topology for MoE inference that cuts hop count from 16 to 7), and the 3D torus Google still uses for training.

Why is opti...

Matt Steiner, VP of Monetization Infrastructure, Ranking & AI Foundations at Meta, walks through how Meta's ad system actually works, and why the infrastructure behind it differs from what you'd build for LLMs.

We cover Andromeda (retrieval on a custom NVIDIA Grace Hopper SKU Meta co-designed), Lattice (consolidating N ranking models into one), GEM (Meta's Generative Ads Recommendation foundation model), and the adaptive ranking model, a roughly one-trillion-parameter recommender served at sub-second latency.

We get into why recommender workloads aren't embarrassingly parallel like LLMs (the "personalization blob"), what that means for Meta's MTIA custom silicon roadmap...

Austin and Vik discuss Credo's acquisition of Dust Photonics, XPO as the new standard for scale-out (maybe instead of CPO?) and some thoughts about Nuvacore entering the CPU scene for agentic AI.

Gavin Baker's tweet: https://x.com/GavinSBaker/status/2044410644301046031?s=20

Vik's Substack: https://www.viksnewsletter.com
Austin's Substack: https://www.chipstrat.com

Chapters

00:00 Introduction to the Semiconductor Landscape
02:49 The Rise of Nuvacore and CPU Innovations
05:27 The Demand for CPUs in the AI Era
07:59 Photonics: The Next Frontier in Semiconductors
10:26 Credo's Acquisition of Dust Photonics
13:12...

In this episode, Austin and Vik discuss if Intel is finally back with CPU partnerships with Google, and heterogeneous inference with SambaNova, while market cap soars above $300B. Vik tries to get his OpenClaw instance to dream every night.

Chapters

00:00 Anthropic's New Direction: Chip Development
02:30 Navigating Subscription Changes and Token Costs
05:25 Exploring Alternative AI Models
08:10 The Economics of AI: Rent vs. Buy
10:56 Intel's Resurgence and Market Dynamics
15:23 Intel's Strategic Partnerships and Market Positioning
19:37 The Role of IPUs in Modern Computing
25:08 Coexistence of x86 and ARM Architectures
29:55 Innovations...

Reiner Pope is the co-founder and CEO of MatX, the startup building chips designed from first principles for LLMs. Before MatX, Reiner was on the Google Brain team training LLMs, and his co-founder Mike Gunter was on the TPU team. They left Google one week before ChatGPT was released.

A counterintuitive throughput insight from the conversation:

“Low latency means small batch sizes. That is just Little’s law. Memory occupancy in HBM is proportional to batch size. So you can actually fit longer contexts than you could if the latency were larger. Low latency is not...

Intel Foundry just partnered with Elon Musk’s Terafab. What is Terafab anyway, why vertically integrated fabs make sense but the economics don’t (yet!), and what Intel is doing here (hint: no idea).

Then: OpenAI acquires TBPN for an estimated $100-300M. Not sure why, but the more interesting thing is the value of niche audiences when five companies control a trillion dollars in AI capex.

And finally, Citrini Research sent an analyst to the Strait of Hormuz with a Pelican case full of spy gear, $15K cash, and Cuban cigars. The most unhinged rese...

In this episode, Austin and Vik analyze NVIDIA's $2 billion investment in Marvell NVLink Fusion, exploring its implications for AI infrastructure, interconnect protocols, and the broader chip ecosystem. They also discuss the current memory market surge, DRAM pricing, and Intel's strategic fab buyback, providing deep insights into industry trends and future directions.

On Substack
Vik: https://www.viksnewsletter.com/
Austin: https://www.chipstrat.com/

Chapters

00:00 NVIDIA's $2 Billion Investment in Marvell
20:11 The Memory Market Crisis
20:16 The Future of Memory Pricing and Consumer Impact
22:55 The Cycle of Supply and Demand in...

In this episode, Austin and Vik analyze recent developments in GloFo patent lawsuits, the impact of TurboQuant on AI inference, and ARM's strategic move into silicon for agentic AI workloads.

Read Vik's substack: https://www.viksnewsletter.com
Read Austin's substack: https://www.chipstrat.com

Chapters

00:00 Patent Wars in Semiconductor Industry
07:14 Understanding TurboQuant and Its Implications
24:42 Innovations in Memory Management
28:00 The Rise of ARM AGI CPUs
32:56 Agentic AI and CPU Compatibility
39:54 Performance Metrics in Agentic AI
44:52 ARM's Market Timing and Challenges

Austin and Vik break down a packed week in semiconductors, covering GTC, OFC, and Micron earnings. The conversation kicks off with Jensen Huang's bold claim that engineers should spend $250K/year on AI tokens, and whether companies will buy tokens or token generators (i.e., on-prem hardware like the Dell Pro Max with GB300). They dig into the CapEx vs OpEx tradeoffs, data security concerns, and how sharing GPU resources might end up looking a lot like the old EDA license model.

Next up: Micron crushed earnings and appears to be designed into Vera Rubin for HBM4 — de...

Vik and Austin unpack the Nvidia GTC keynote with fresh, top-of-mind takes while trying to breakdown key announcements, what matters and what doesn't. They discuss Groq's LPX, optics+copper for scale up, new CPU requirements, CPO for networking, and what agents means for software, and much, much, more.

Check out Austin's substack: https://www.chipstrat.com
Check out Vik's substack: https://www.viksnewsletter.com

Chapters

00:00 Introduction and Keynote Context
03:18 Keynote Highlights and Gaming Innovations
06:18 Generative AI: The Three Eras
09:28 Inference: The New Revenue Generator
12:21 NVIDIA's Tiered...

Austin recaps moderating an agentic AI panel at Synopsys Converge, then gives an in-depth technical breakdown of Meta's MTIA custom silicon. Why they're building it, how chiplets let them ship a new chip every 6 months, and how the roadmap is shifting toward gen AI inference. Vik digs into Applied Optoelectronics (AAOI), the vertically integrated Texas laser shop whose stock went from $1.48 to $100+, and whether history is about to rhyme.                     

Austin Lyons: https://www.chipstrat.com
Vik Sekar: https://www.viksnewsletter.com/
                                                                                                                                  
Topics covered:
• Agentic AI in chip design — how it changes roles for junior and senior engineers
• Optical circuit switching and what it means for...

This week, Austin and Vik break down the optics vs. copper debate that rocked semis this week. Nvidia dropped $4 billion on Lumentum and Coherent, Credo posted a blowout quarter betting on copper, and then Hock Tan shocked everyone claiming 400G per lane works over copper in Broadcom’s labs — potentially pushing CPO out to 2030+. Plus, Vik’s 4D chess conspiracy theory on why Hock Tan is talking up copper when Broadcom is a CPO company.

Like, subscribe, and drop your thoughts on the copper vs. optics debate in the comments!

Subscribe to our newsletters:
* Chipst...

This week, we move from optics technology to optics companies. We walk the AI optical supply chain from bottom to top. Main debate: Who has a moat? Who is already priced for perfection?  *Not investment advice, do your own due diligence*

AXTI - Indium phosphide substrate supplier. Critical bottleneck in the laser stack. Major China export-control risk. Massive stock run vs thin earnings.

Tower Semiconductor - Leading silicon photonics foundry. 5x capacity expansion with customer prepayments. Strong process lock-in. Pure-play optics exposure.

GlobalFoundries - 300mm monolithic photonics p...

Austin and Vik delve into the evolving landscape of optics and networking, particularly in relation to AI and data centers.

The conversation covers various scales of networking, including scale across, scale out, and scale up, while also addressing the demand-supply dynamics in laser manufacturing and the future of optical circuit switches.

The episode highlights the technological advancements and market opportunities in the optics sector, emphasizing the significance of these developments for the future of AI.

Takeaways

In this episode of the Semi Doped podcast, Austin and Vik delve into the current state of the semiconductor industry, focusing on the memory crisis driven by increasing demand from AI applications. They discuss the implications of rising memory prices, the impact of hyperscaler spending on the market, and the strategic moves of major players like Google, Microsoft, Meta, and Amazon in the AI landscape.

Takeaways

In this episode, Vik and Wayne Nelms discuss the emerging financial exchange for GPU compute, exploring its implications for the AI infrastructure market. They discuss the value of compute, pricing dynamics, hedging strategies, and the future of GPU and memory trading. 

Wayne shares insights on partnerships, the depreciation of GPUs, and how inference demand may reshape hardware utilization. The conversation highlights the importance of financial products in facilitating data center development and optimizing profitability in the evolving landscape of compute resources.

Takeaways

Vik and Val Bercovici discuss the evolution of storage solutions in the context of AI, focusing on Weka's innovative approaches to context memory, high bandwidth flash, and the importance of optimizing GPU usage.

Val shares insights from his extensive experience in the storage industry, highlighting the challenges and advancements in memory requirements for AI models, the significance of latency, and the future of storage technologies.

Takeaways

Austin and Vik discuss the emerging trend of AI agents, particularly focusing on Claude Code and OpenClaw, and the resulting hardware implications.

Key Takeaways:

Maia 100 was a pre-GPT accelerator.
Maia 200 is explicitly post-GPT for large multimodal inference.

Saurabh Dighe says if Microsoft were chasing peak performance or trying to span training and inference, Maia would look very different. Higher TDPs. Different tradeoffs. Those paths were pruned early to optimize for one thing: inference price-performance. That focus drives the claim of ~30% better performance per dollar versus the latest hardware in Microsoft’s fleet.

Intereting topics include:
• What “30% better price-performance” actually means
• Who Maia 200 is built for
• Why Microsoft bet on inference when designing Maia back in 2022...

OpenAI's partnership with Cerebras and Nvidia's announcement of context memory storage raises a fundamental question: as agentic AI demands long sessions with massive context windows, can SRAM-based accelerators designed before the LLM era keep up—or will they converge with GPUs?

Key Takeaways
1. Context is the new bottleneck. As agentic workloads demand long sessions with massive codebases, storing and retrieving KV cache efficiently becomes critical.
2. There's no one-size-fits-all. Sachin Khatti's (OpenAI, ex-Intel) signals a shift toward heterogeneous compute—matching specific accelerators to specific workloads.
3. Cerebras has 44GB of S...