Side-Stacked Memory Chips Could Solve HBM's Heat Problem
V-Die and MOSAIC arrange DRAM dies side by side instead of stacking them vertically, aiming to solve HBM's heat problem and boost memory bandwidth for AI chips.
Today's high-bandwidth memory (HBM) stacks DRAM dies vertically atop a base die, but as AI models keep growing, this approach is running into thermal and capacity limits — adding more layers risks trapping heat inside the stack. Two research groups presented at the IEEE VLSI Symposium propose an alternative: arranging DRAM dies side by side rather than on top of each other.
South Korea's UNIST and Hanbat National University team built V-Die, which uses microfluidic cooling channels between horizontally arranged dies to keep temperatures around 45°C, while eliminating vertical through-silicon vias to free up space for more memory cells and denser connections. Simulations show an 82 percent speed improvement over HBM4 and notably lower latency on GPT3-scale workloads.
Japan's University of Tokyo, Tohoku University, and Riken team developed MOSAIC, which connects dies to the substrate using inductive coupling coils instead of direct electrical bonds, giving far more tolerance for alignment errors. MOSAIC is expected to double HBM4's memory capacity while raising peak temperature by only about 1°C.
Both designs remain at the prototype stage, but they signal a possible shift in how chipmakers approach memory stacking to overcome the thermal and bandwidth ceilings that current HBM architectures are approaching — a shift with direct implications for future AI hardware design.