NVHBM Redefines AI Chip Architecture: From Compute-Centric Design to Memory-Centric AI
NVHBM Redefines AI Chip Architecture: From Compute-Centric Design to Memory-Centric AI¶
- Source: SemiVision
- Author: SEMIVISION
- Published: 2026-08-28T18:32:18+08:00
- URL: https://tspasemiconductor.substack.com/p/nvhbm-redefines-ai-chip-architecture
中文翻譯與英文原文（雙語）¶
(2) NVHBM 重新定義 AI 晶片架構：從運算為中心設計邁向記憶體為中心的 AI --- (2) NVHBM Redefines AI Chip Architecture: From Compute-Centric Design to Memory-Centric AI
NVHBM Redefines AI Chip Architecture: From Compute-Centric Design to Memory-Centric AI¶
NVHBM 重新定義 AI 晶片架構：從運算為中心設計邁向記憶體為中心的 AI
Original Article By SemiVision Research [Reading time: 10 mins]¶
原文作者：SemiVision Research [閱讀時間：10 分鐘]
Aug 28, 2026
∙ Paid∙ 付費
Share分享
HBM Base Die Is Evolving from an I/O Layer into the Control Plane of AI Memory
HBM 基底晶片正從 I/O 層演進為 AI 記憶體的控制平面
The role of the HBM base die is undergoing a structural transformation. It is no longer simply a passive interface layer at the bottom of the HBM stack responsible for TSV routing, PHY functions, and signal connectivity. Instead, it is evolving toward alogic-rich base die, custom memory controller die, and eventually an AI memory management die.
HBM 基底晶片的角色正經歷結構性轉變。它不再只是位於 HBM 堆疊底部、負責 TSV 繞線、PHY 功能與訊號連接的被動介面層；相反地，它正朝向邏輯密集型的基底晶片、客製化記憶體控制器晶片，並最終發展為 AI 記憶體管理晶片演進。
The key driver behind this transition is that the bottleneck in AI systems is shifting away from pure compute performance towarddata movement, memory bandwidth, memory capacity, power efficiency, and ultimately cost per token.
推動此轉變的關鍵驅動力在於：AI 系統的瓶頸已從純粹的運算效能，轉向資料搬移、記憶體頻寬、記憶體容量、電源效率，乃至於每 token 的成本。
Traditional GPU architectures are fundamentally XPU-centric. The GPU or ASIC performs the computation while also controlling the memory controller, memory scheduling, cache hierarchy, and data orchestration. HBM primarily serves as a high-bandwidth memory resource. This architecture remains highly effective for training because model parameters are constantly updated and the system requires maximum flexibility.
傳統 GPU 架構本質上是以 XPU 為中心。GPU 或 ASIC 不僅執行運算，還負責控制記憶體控制器、記憶體排程、快取階層以及資料協調。HBM 主要作為高頻寬記憶體資源。此架構在訓練階段依然極為有效，因為模型參數需持續更新，且系統需要最大的靈活性。
However, as AI moves toward large-scale inference, the bottleneck changes. Weights, KV cache, activations, and context data must constantly move across different levels of the memory hierarchy. Simply adding more FLOPS no longer translates linearly into higher token throughput. Increasingly, system efficiency depends onwhere data resides, when it moves, how much data needs to move, and whether that movement is necessary at all.
然而，隨著 AI 邁向大規模推論，瓶頸已發生轉變。權重、KV 快取、活化值及上下文資料必須在不同層級的記憶體階層間持續流動。單純增加 FLOPS 已無法線性提升 token 吞吐量。系統效率日益取決於資料位於何處、何時移動、需移動多少資料，以及該移動是否必要。
AI architecture is therefore gradually shifting from acompute-centric architecture toward a data-centric architecture, and the HBM base die sits directly at the boundary between compute and memory.
因此，AI 架構正逐漸從以運算為中心轉向以資料為中心，而 HBM 基底晶粒（base die）正處於運算與記憶體的交界處。
HBM4 Marks the Beginning of the Logic-Rich Base Die Era
HBM4 標誌著邏輯豐富型基底晶粒時代的開端
HBM4 represents an important inflection point.
HBM4 代表了一個重要的轉折點。
As the HBM interface expands to 2,048 bits, the I/O, power delivery, signal integrity, and control complexity handled by the base die increase substantially. This creates a stronger need for advanced logic processes and allows the base die to move beyond PHY, TSV routing, and basic control functions.
隨著 HBM 介面擴增至 2,048 位元，基底晶粒（base die）所處理的 I/O、電源傳輸、訊號完整性與控制複雜度大幅增加。這使得對先進邏輯製程的需求更為迫切，並讓基底晶粒得以超越 PHY、TSV 繞線及基本控制功能。
Over time, additional functions could migrate into the base die, including ECC, RAS, address remapping, local memory scheduling, refresh management, data movement engines, prefetching, compression, and potentially certain data-placement functions.
隨時間推移，更多功能可能遷移至基底晶粒，包括 ECC、RAS、位址重新映射、本端記憶體排程、刷新管理、資料移動引擎、預取、壓縮，乃至某些資料放置功能。
As a result, the definition of HBM itself begins to change.
因此，HBM 本身的定義也開始發生改變。
The traditional architecture can be simplified as:
傳統架構可簡化為：
DRAM Die + TSV + Base Die + Packaging
DRAM 晶粒 + TSV + 基底晶粒 + 封裝
The future architecture increasingly resembles:
未來架構則日益趨近於：
DRAM Stack + Advanced Logic Base Die + Memory IP + Controller + Firmware + Advanced Packaging + System Co-design
DRAM 堆疊 + 先進邏輯基底晶粒 + 記憶體 IP + 控制器 + 韌體 + 先進封裝 + 系統共同設計
HBM is therefore evolving from a pure memory product into alogic-plus-memory system product.
因此，HBM 正從純粹的記憶體產品演進為結合邏輯與記憶體的系統級產品。
This transition is important because future HBM value creation may no longer be determined only by capacity, stack height, and bandwidth. The amount of logic and system functionality integrated into the base die could become another major source of differentiation and value.
此轉型至關重要，因為未來 HBM 的價值創造可能不再僅取決於容量、堆疊高度與頻寬。整合至基底晶粒（base die）中的邏輯電路與系統功能，將成為另一項主要的差異化來源與價值驅動因素。
Memory Controllers Will Be Partitioned Rather Than Simply Relocated
記憶體控制器將進行分割，而非單純遷移
The future architecture should not be interpreted as the entire HBM memory controller simply moving from the GPU into the base die.
未來的架構不應被解讀為將整個 HBM 記憶體控制器單純從 GPU 移至基底晶粒。
A more realistic evolution ismemory controller partitioning.
更符合實際的演進方向是記憶體控制器的分割。
The XPU would continue to handle global memory management functions such as workload scheduling, coherency, QoS, global address management, and overall memory allocation.
XPU 將繼續負責全域記憶體管理功能，例如工作負載排程、一致性維護、服務品質（QoS）、全域位址管理以及整體記憶體配置。
The HBM base die, meanwhile, could increasingly handle functions closer to the physical DRAM, including local scheduling, ECC, refresh management, row management, address translation, data rearrangement, and reliability management.
與此同時，HBM 基底晶粒可逐步承擔更貼近實體 DRAM 的功能，包括區域排程、錯誤校正碼（ECC）、刷新管理、列管理、位址轉換、資料重排以及可靠性管理。
The architecture could therefore evolve toward:
因此，架構可望朝以下方向演進：
XPU = Global Memory Brain
XPU = 全域記憶體大腦
HBM Base Die = Local Memory Brain
HBM 基底晶片 = 區域記憶體大腦
This architecture becomes particularly valuable in the inference era.
此架構在推論時代尤具價值。
When large volumes of KV cache, weights, and activation requests simultaneously access HBM, requiring every request to return to a centralized XPU memory controller creates additional command and control overhead.
當大量 KV 快取、權重與活化請求同時存取 HBM 時，若要求每個請求皆回傳至集中式的 XPU 記憶體控制器，將產生額外的指令與控制開銷。
If the base die can perform request reordering, prefetching, compression, local scheduling, or data filtering, the system can improveeffective bandwidthwithout necessarily increasing physical bandwidth.
若基底晶粒能執行請求重排序、預取、壓縮、本地排程或資料過濾，系統即可在不必然增加實體頻寬的情況下提升有效頻寬。
HBM would therefore gradually evolve frompassive memory toward semi-active memory.
因此，HBM 將逐漸從被動記憶體演進為半主動記憶體。
cHBM Turns the Base Die into the Customization Layer
cHBM 將基底晶粒轉變為客製化層
Different AI workloads have fundamentally different memory requirements.
不同的 AI 工作負載具有本質上相異的記憶體需求。
Training requires large-scale tensor movement, parameter updates, and collective communication. LLM inference places greater emphasis on weight streaming, KV cache capacity, and memory bandwidth. Recommendation systems require large-scale embedding lookups, while multimodal and video models exhibit very different memory-access patterns.
訓練需要大規模的張量移動、參數更新與集體通訊。LLM 推論則更著重於權重串流、KV 快取容量與記憶體頻寬。推薦系統需要大規模的嵌入查找，而多模態與視訊模型則呈現截然不同的記憶體存取模式。
Using exactly the same HBM memory architecture for every XPU is therefore unlikely to deliver optimal efficiency.
因此，對每種 XPU 採用完全相同的 HBM 記憶體架構，不太可能實現最佳效率。
The logical direction for custom HBM, or cHBM, is to maintain a relatively standardized DRAM stack while customizing the logic base die for specific XPUs and workloads.
客製化 HBM（cHBM）的合理方向是維持相對標準化的 DRAM 堆疊，同時針對特定 XPU 與工作負載客製化邏輯基底晶粒。
The architecture can be summarized as:
該架構可總結如下：
Standard DRAM Stack + Custom Logic Base Die
標準 DRAM 堆疊 + 自訂邏輯基底晶粒
This is fundamentally similar to the chiplet concept.
這在根本上與小晶片（chiplet）概念相似。
The DRAM stack becomes a standardizedmemory chiplet, while the base die becomes amemory I/O and logic chiplet.
DRAM 堆疊成為標準化的記憶體小晶片，而基底晶粒則成為記憶體 I/O 與邏輯小晶片。
As a result, one of the most important questions in the future HBM ecosystem will no longer be only who has the highest capacity, tallest stack, or highest bandwidth.
因此，未來 HBM 生態系中最重要的問題之一，將不再僅是誰擁有最高容量、最高堆疊或最高頻寬。
The more strategic question becomes:
更具戰略意義的問題在於：
Who controls the HBM base-die IP?
誰掌控 HBM 基底晶片（base-die）的知識產權（IP）？
The company controlling the base-die architecture could gain influence over data movement, memory scheduling, memory hierarchy, and ultimately the interface between the XPU and memory subsystem.
掌握基底晶片架構的公司，將能影響資料移動、記憶體排程、記憶體階層，乃至於 XPU 與記憶體子系統之間的介面。
AI Memory Competition Is Shifting from Bandwidth to Data Placement
AI 記憶體競爭正從頻寬轉向資料放置
HBM’s greatest advantage is bandwidth, but its major structural disadvantages arecost per bit and capacity economics.
HBM 最大的優勢在於頻寬，但其主要的結構性缺點則是每位元成本與容量經濟效益。
As AI models continue to scale, storing all weights, KV cache, context, and other data in expensive HBM becomes economically unrealistic.
隨著 AI 模型持續擴大規模，將所有權重、KV cache、上下文及其他資料全部儲存於昂貴的 HBM 中，在經濟上已變得不切實際。
Future AI servers are therefore likely to develop a much more explicit memory hierarchy:
因此，未來的 AI 伺服器很可能會發展出更為明確的記憶體階層架構：
On-die SRAM → HBM → High-Bandwidth Flash → CXL DRAM → CXL SSD / Flash → Storage
片上 SRAM → HBM → 高頻寬 Flash → CXL DRAM → CXL SSD / Flash → 儲存裝置
Once this hierarchy becomes established, the critical question is no longer simply:
一旦此層級架構確立，關鍵問題便不再只是：
“How much HBM does the accelerator have?”
「加速器擁有多少 HBM？」
The more important question becomes:
更重要的問題變為：
Which data actually deserves to stay in HBM?
哪些數據實際上應該保留在 HBM 中？
Hot data can remain in SRAM or HBM. Warm data can migrate toward lower-cost memory tiers, while cold data can move further toward flash or storage.
熱數據可以保留在 SRAM 或 HBM 中。溫數據可以遷移至成本較低的記憶體層級，而冷數據則可進一步移往快閃記憶體或儲存裝置。
The key architectural capability therefore becomesdata placement.
因此，關鍵的架構能力在於數據放置。
If the HBM base die can cooperate with the XPU, CXL controller, and other memory tiers, it could potentially help determine where data should reside based on access frequency, latency sensitivity, bandwidth requirements, and data reuse rates.
若 HBM 基底晶粒能與 XPU、CXL 控制器及其他記憶體層級協同運作，便有可能依據存取頻率、延遲敏感度、頻寬需求及數據重複使用率，協助判斷數據應駐留於何處。
At that point, the base die begins evolving beyond a memory controller into a:
此時，基底晶粒開始從記憶體控制器演變為：
Memory Orchestrator.記憶體協調器。
Token Economics Will Redefine HBM KPIs
代幣經濟學將重新定義 HBM 的關鍵績效指標
An AI system ultimately exists to generate useful AI output, not simply to maximize TB/s.
AI 系統的最終目標是產生有用的 AI 輸出，而非單純追求最大化 TB/s。
This means the marginal benefit of continuously increasing physical HBM bandwidth will eventually decline.
這意味著持續提升實體 HBM 頻寬的邊際效益最終將會遞減。
A system may provide extremely high theoretical HBM bandwidth, but if a significant portion of that bandwidth is consumed by duplicate transfers, cache misses, data rearrangement, refresh overhead, incorrect prefetching, or low-value data movement, the effective bandwidth converted into token throughput may be substantially lower.
一個系統可能提供極高的理論 HBM 頻寬，但若其中很大一部分頻寬被重複傳輸、快取未命中、資料重排、重新整理開銷、錯誤預取或低價值資料移動所消耗，則轉換為 token 吞吐量的有效頻寬可能會大幅降低。
Conversely, if the base die can improve memory utilization through scheduling, compression, prefetching, data filtering, KV cache management, and intelligent data placement, token throughput could increase even without increasing physical bandwidth.
反之，若基底晶片能透過排程、壓縮、預取、資料過濾、KV 快取管理與智慧資料放置來提升記憶體利用率，即使不增加實體頻寬，也能提高 token 吞吐量。
Future HBM KPIs could therefore expand beyond traditional metrics such as:
因此，未來的 HBM 關鍵績效指標（KPI）可能會擴展至超越傳統指標的範疇，例如：
GB CapacityGB 容量
TB/s
Gbps per Pin每針腳 Gbps
toward metrics such as:朝向以下指標：
Effective Bandwidth有效頻寬
Useful Bandwidth / Physical Bandwidth
有用頻寬／實體頻寬
Tokens per HBM GB每 HBM GB 的 Token 數量
Tokens per Joule每焦耳 Token 數
Tokens per Watt每瓦特代幣數
Cost per Token每個代幣的成本
This represents a fundamental architectural transition.
這代表了一項根本性的架構轉變。
The question is no longer simply whether more HBM improves performance.
問題已不再單純是更多的 HBM 是否能提升效能。
Instead, hyperscalers and accelerator designers increasingly need to ask:
相反，超大型雲端服務供應商與加速器設計者日益需要思考：
How many useful tokens can each expensive HBM stack generate?
每一組昂貴的 HBM 堆疊能產生多少有用的 token？
The HBM Base Die Will Become the Intersection of Foundry, Memory, and ASIC Ecosystems
HBM 基底晶片將成為晶圓代工、記憶體與 ASIC 生態系的交匯點
During the HBM3E generation, much of the value remained concentrated in DRAM technology, TSV, stacking, and packaging.
在 HBM3E 世代，大部分價值仍集中於 DRAM 技術、TSV、堆疊與封裝。
With HBM4, HBM4E, and cHBM, the importance of the logic base die increases rapidly. The supply chain therefore begins transitioning from a relatively traditional memory supply chain toward an ecosystem involving:
隨著 HBM4、HBM4E 與 cHBM 的出現，邏輯基底晶粒的重要性迅速提升。因此，供應鏈開始從相對傳統的記憶體供應鏈，轉向涵蓋以下環節的生態系：
Memory Vendors + Foundries + ASIC/IP Vendors + Advanced Packaging
記憶體廠商 + 晶圓代工廠 + ASIC/IP 供應商 + 先進封裝
Memory vendors manufacture the DRAM stack.
記憶體廠商負責製造 DRAM 堆疊。
Foundries manufacture the advanced logic base die.
晶圓代工廠製造先進邏輯基底晶片。
GPU, ASIC, and IP companies provide memory controllers, PHYs, connectivity IP, and custom logic.
GPU、ASIC 與 IP 公司提供記憶體控制器、實體層（PHY）、連接性 IP 及客製化邏輯。
The complete product is then integrated through HBM stacking, bonding, testing, and 2.5D/3D advanced packaging.
最終產品透過 HBM 堆疊、接合、測試以及 2.5D/3D 先進封裝技術完成整合。
Future HBM competition will therefore no longer be determined solely by competition among SK hynix, Samsung, and Micron in DRAM technology.
因此，未來 HBM 的競爭將不再僅取決於 SK hynix、Samsung 與 Micron 在 DRAM 技術上的較量。
The competitive landscape expands into:
競爭格局擴展至：
DRAM Technology × Logic Process × Controller IP × Advanced Packaging × System Architecture
DRAM 技術 × 邏輯製程 × 控制器 IP × 先進封裝 × 系統架構
This could fundamentally redistribute value across the HBM supply chain.
這可能從根本上重新分配 HBM 供應鏈的價值。
The Base Die Could Ultimately Become an AI Memory Management Die
基底晶片最終可能演變為 AI 記憶體管理晶片
The evolution of the HBM base die over the next several generations can be summarized as a four-stage roadmap.
HBM 基底晶粒在未來幾個世代的演進，可歸納為四個階段的發展路線圖。
HBM3E: I/O Base DieHBM3E：I/O 基底晶粒
The base die primarily handles PHY, TSV routing, I/O, and basic control functions.
基底晶粒主要負責處理 PHY、TSV 繞線、I/O 及基本控制功能。
HBM4: Logic Base DieHBM4：邏輯基底晶粒
More advanced logic processes enable additional memory-management and reliability functions.
更先進的邏輯製程可實現額外的記憶體管理與可靠性功能。
HBM4E / cHBM: Custom Memory Controller Die
HBM4E / cHBM：自訂記憶體控制器晶片
The base die becomes increasingly customized for specific XPUs, ASICs, hyperscalers, and AI workloads.
基底晶片日益針對特定 XPU、ASIC、超大型業者及 AI 工作負載進行客製化。
Next-Generation HBM: AI Memory Management Die
次世代 HBM：AI 記憶體管理晶片
The base die could eventually incorporate data movement engines, compression, prefetching, KV cache management, security, telemetry, RAS, coherency functions, and potentially certain forms of near-memory acceleration.
基底晶片最終可能整合資料移動引擎、壓縮、預取、KV 快取管理、安全性、遙測、RAS（可靠性、可用性與可維護性）、一致性功能，以及潛在的某種形式之近記憶體加速技術。
At that stage, the base die is no longer simply “the die underneath the HBM stack.”
在該階段，基底晶粒不再僅僅是「HBM 堆疊下方的晶粒」。
It becomes an important control node for the entire AI memory hierarchy.
它成為整個 AI 記憶體階層的關鍵控制節點。
The long-term evolution of HBM can therefore be summarized in one sentence:
因此，HBM 的長期演進可歸結為一句話：
HBM Base Die is evolving from a physical I/O layer into the control plane of the AI memory hierarchy.
HBM 基底晶片（Base Die）正從實體 I/O 層演變為 AI 記憶體階層的控制平面。
The next phase of competition will increasingly shift away from simply determining who can deliver the highest HBM bandwidth. The more important question will be who can move the right data to the right memory tier at the right time, using the lowest possible amount of energy, while maximizing useful token output.
下一階段的競爭將逐漸不再僅聚焦於誰能提供最高的 HBM 頻寬。更關鍵的問題在於：誰能以最低的能耗，在正確的時間將正確的資料移至正確的記憶體層級，同時最大化有效 token 輸出。
Ultimately, the strategic value of the HBM base die could become substantially greater than what is implied by today’s relatively narrow discussion around HBM4 base-die foundry orders. It is becoming one of the key architectural layers connectingcompute, memory, connectivity, advanced packaging, and token economicsin next-generation AI systems.
最終，HBM 基底晶片的戰略價值可能遠超當前圍繞 HBM4 基底晶片晶圓代工訂單的狹隘討論。它正成為下一代 AI 系統中連結運算、記憶體、連線能力、先進封裝與代幣經濟的關鍵架構層。
Share分享
Previous上一頁Next下一則