Memory Is the New Compute: Why SK Hynix’s Investment in Semidynamics Signals a Structural Shift in AI Architecture
Memory Is the New Compute: Why SK Hynix’s Investment in Semidynamics Signals a Structural Shift in AI Architecture¶
- Source: SemiVision
- Author: SEMIVISION
- Published: 2026-04-15T20:20:58+08:00
- URL: https://tspasemiconductor.substack.com/p/memory-is-the-new-compute-why-sk (1) Memory Is the New Compute: Why SK Hynix’s Investment in Semidynamics Signals a Structural Shift in AI Architecture
Memory Is the New Compute: Why SK Hynix’s Investment in Semidynamics Signals a Structural Shift in AI Architecture¶
Original Article By SemiVision Research [Reading time: 8 mins]¶
Apr 15, 2026
∙ Paid
Memory Is the New Compute: Why SK Hynix’s Investment in Semidynamics Signals a Structural Shift in AI Architecture
From a Simple Investment to a Structural Signal
When SK Hynix announced a strategic investment in Spanish RISC-V startup Semidynamics, the headline appeared straightforward: a memory company backing an emerging AI chip designer. But beneath the surface, this move reflects something far more fundamental—a shift in how the industry understands AI infrastructure itself. For over a decade, AI scaling has been framed as a compute problem, dominated by GPUs, FLOPS, and parallel processing power. That paradigm is now breaking down. What this partnership reveals is that AI, particularly in the inference era, is increasingly constrained not by how fast we can compute, but by how efficiently we can move and access data. In other words, memory is no longer a supporting component—it is becoming the defining bottleneck and, increasingly, the primary design axis.
2月5日
The Real Bottleneck: AI Inference Is Memory-Bound
The transition from training to inference fundamentally changes system requirements. Training workloads are compute-intensive and benefit from dense matrix operations, but inference—especially with large language models, long-context reasoning, and multi-agent systems—is dominated by memory access patterns. Tokens must continuously retrieve context, parameters must be streamed efficiently, and latency becomes tightly coupled with data movement rather than arithmetic throughput. The industry’s current answer has been to rely heavily on high-bandwidth memory (HBM), a solution that works but introduces structural constraints: extremely high cost, severe supply limitations, and poor scalability at system level. As a result, even as compute continues to scale, AI systems remain constrained by memory bandwidth, capacity, and availability. This is precisely the gap that Semidynamics is targeting—not by competing on faster memory, but by rethinking how systems operate under memory constraints.
A Different Path: Architecture Instead of Bandwidth
Semidynamics’ core innovation lies in its Gazzillion memory subsystem, which takes a fundamentally different approach from traditional CPU or GPU architectures. Conventional designs rely heavily on cache hierarchies and assume high cache hit rates; when data is not found in cache, the system stalls, waiting for memory access.
GPUs mitigate this by using extremely fast memory like HBM to reduce latency impact. Semidynamics, however, challenges this assumption entirely. Instead of waiting for data, its architecture continues executing, effectively hiding memory latency at the system level. This means performance can be maintained even when using slower, non-HBM memory. The implication is profound: rather than making memory faster—which is increasingly expensive and supply-constrained—the system becomes tolerant to slower memory. This shifts the optimization problem from materials and process scaling to architectural intelligence.
Gazzillion Misses is Semidynamics’ architectural approach to one of the hardest problems in modern computing: how to keep performance high when workloads constantly miss cache and must fetch data from slower, farther memory. In traditional CPU designs, frequent cache misses can stall execution and severely reduce efficiency. Gazzillion Misses is designed to change that equation. Instead of assuming that most useful data must stay close to the processor, it allows the system to continue making progress even when memory accesses are irregular, sparse, and latency-heavy.
This matters because many of today’s important workloads no longer behave like classic desktop or server applications. Machine learning, recommendation systems, key-value stores, and sparse data or HPC applications often depend on huge datasets that cannot fit neatly into cache. Their access patterns are unpredictable, memory-intensive, and difficult for conventional architectures to optimize. Gazzillion Misses targets exactly these conditions.
The idea is especially relevant in the AI era. As models grow larger and inference becomes more memory-bound, overall system efficiency depends not only on compute throughput, but on how well the architecture handles data movement. By reducing the performance penalty of cache misses, Gazzillion Misses aims to unlock better utilization of non-HBM memory, improve scalability, and enable more flexible, memory-centric AI and data infrastructure.
Gazzillion Misses is especially useful for SoCs because these chips often face strict area, power, and memory constraints. In designs with limited SRAM or cache, performance usually drops when data cannot stay close to the core. Gazzillion Misses helps reduce that penalty by allowing computation to continue efficiently even when memory accesses go farther out. It is also well suited for high-bandwidth and streaming architectures, where large amounts of data move continuously through the system and conventional cache-centric designs become less effective. In SoCs with vector units, the benefit can be even greater, because vector engines consume data quickly and can be limited by memory latency rather than raw compute capability.
By improving tolerance to cache misses and irregular access patterns, Gazzillion Misses enables better utilization of compute resources, supports more scalable data movement, and helps SoCs maintain higher performance without relying entirely on massive on-chip cache or expensive memory subsystems.
Challenging NVIDIA’s Core Assumption
Today’s AI infrastructure is implicitly governed by NVIDIA, not through formal standards but through system architecture and deployment scale. NVIDIA’s approach has been clear and effective: overcome memory bottlenecks by increasing bandwidth, primarily through HBM integration. This strategy has enabled rapid scaling but at the cost of power consumption, system complexity, and supply chain pressure. Semidynamics represents a counter-thesis. Instead of scaling memory performance linearly with compute, it questions whether the dependency on ultra-fast memory is necessary at all. This creates a divergence in architectural philosophy: one path doubles down on bandwidth, while the other redefines how systems behave under latency. SK Hynix’s involvement is critical here—not just as a supplier of memory, but as a participant in shaping which architectural direction will define the next decade of AI systems.
2月26日
RISC-V and the Reinvention of AI ISA
Semidynamics’ RISC-V strategy is not just about adopting an open architecture. It is about building a vector-processing foundation better suited for AI-native workloads. Its VPU implements the RISC-V Vector 1.0 specification, including key features such as integer multiply and segmented loads, while omitting vector atomics that fall outside the current 1.0 scope. More importantly, the design is ready for out-of-order (OOO) support, including register renaming, giving it a path toward higher-performance execution as workloads become more complex.
The architecture is also highly configurable. Semidynamics allows vector register length (VLEN) to scale from 128b to 4096b, while data path width can be configured from 128b to 512b. It also includes a fast cross-lane network to accelerate operations such as slide, gather, compress, and expand—functions that are increasingly important in AI, data movement, and irregular vector workloads. Together, these features show that Semidynamics is not simply implementing RISC-V for compatibility, but is shaping it into a flexible AI-oriented vector platform with room to scale across different performance and system targets.
From Chip to Rack: Competing at the System Level
One of the most telling aspects of Semidynamics’ strategy is that it does not intend to compete at the chip level alone. The company is building full-stack solutions, including SoCs (under the Atrevido brand), system architecture, rack-level infrastructure, and software stacks. This mirrors the playbook that has made NVIDIA dominant: control the system, not just the silicon. As CEO Roger Espasa noted, selling chips alone is no longer sufficient to attract data center customers. Modern AI deployments require integrated solutions that can be deployed seamlessly into infrastructure environments. This reinforces a broader industry trend: competition is no longer about individual components, but about vertically integrated systems that optimize performance, programmability, and total cost of ownership.
Unified Memory: The Missing Layer in AI Infrastructure
Another critical pillar of Semidynamics’ vision is the concept of unified memory at the data center scale. While unified memory has been a cornerstone of CPU architecture for decades, AI systems today remain highly fragmented, with separate memory domains across GPUs, CPUs, and storage layers. This fragmentation introduces inefficiencies in data movement, increases programming complexity, and limits system-level optimization. Semidynamics aims to extend the concept of unified memory across the entire data center, effectively creating a large, coherent memory pool accessible by AI workloads. If realized, this would simplify software development, reduce data transfer overhead, and unlock new levels of system efficiency. It also aligns closely with the needs of emerging AI workloads, which increasingly require flexible, scalable memory access rather than fixed, high-bandwidth channels.
Why Compression Doesn’t Reduce Demand
Recent developments such as TurboQuant and ultra-low precision models (e.g., sub-2-bit representations) have sparked discussions about reducing memory requirements. However, history suggests the opposite outcome. Efficiency gains in computing rarely lead to reduced resource consumption; instead, they enable new use cases and drive higher overall demand. Espasa’s perspective aligns with this pattern: as models become more efficient, they tend to grow larger, incorporate more features, and expand into new applications. Similarly, as memory becomes easier to use or more efficiently utilized, systems scale up rather than down. Compression, therefore, does not eliminate the need for memory—it accelerates its consumption.
Google: TurboQuant: Redefining AI efficiency with extreme compression
The Multi-Agent Future and Exponential Memory Growth
Looking ahead, the rise of multi-agent AI systems introduces another layer of complexity. Instead of a single large model, future applications will involve multiple specialized agents collaborating in real time. Even if individual models become smaller, the aggregate demand for memory and compute will increase significantly. Each agent requires its own context, state, and data access, leading to a multiplication of memory requirements at the system level. This trend reinforces the idea that memory demand is structurally expanding, not contracting. It also explains why SK Hynix projects memory shortages could extend toward the end of the decade. The industry is not facing a temporary imbalance—it is entering a prolonged phase of demand outpacing supply.
Conclusion: The Battlefield Has Shifted
The significance of SK Hynix’s investment in Semidynamics lies not in the size of the deal, but in what it represents. AI infrastructure is undergoing a fundamental shift—from compute-centric scaling to memory-centric architecture. The key questions for the next decade are no longer about who can build the fastest processor, but about who can design systems that use memory most efficiently, minimize data movement, and enable scalable, programmable architectures. Semidynamics represents one possible answer: a system that does not rely on ever-faster memory, but instead redefines how computation interacts with memory. With SK Hynix stepping into this space, memory vendors are no longer just suppliers—they are becoming architects of the AI era.
2月4日
3月2日
PreviousNext
TopLatestDiscussions
See all