Samsung used a graphics card with built-in PIM (processing-in-memory) to create the world’s first large-scale computing system. Citing Business Korea, AI training performance was increased by 2.5 times by using 96 PIM-modified AMD Instinct MI100 GPUs.
PIM is a new type of computer memory that accelerates the processing power of processors such as CPUs and GPUs. the most important feature of PIM is that each memory module is able to process its own data, reducing the amount of data needed to transfer between memory and processor.
Samsung publicly demonstrated PIM-modified GPUs in October this year, and now the company has 96 PIM-modified GPUs in a cluster. Compared with ordinary memory, these improved MI100 chips not only improve performance by 2.5 times, but also reduce power consumption by 2.67 times, greatly improving the efficiency of the GPU when running AI algorithms.
Samsung has been developing PIM for some time. The company demonstrated multiple implementations in 2021 involving several different memory types, including DDR4, LPDDR5X, GDDR6 and HBM2. In the LPDDR5 form factor, Samsung saw a 1.8x performance increase, a 42.6% power reduction and a 70% latency reduction in test programs involving Meta AI workloads.
The AMD Instinct MI100 gas pedal has been completely redesigned to fully embrace the new era of computing.1 The MI100 gas pedal features AMD’s CDNA architecture, which delivers a huge leap forward in compute and interconnect performance, with nearly 3.5x performance improvement in high-performance computing (FP32 matrix) and nearly 7x performance improvement in artificial intelligence (FP16) compared to the previous generation of AMD gas pedals.
Samsung isn’t the only company developing PIM chips — SK Hynix released its own PIM module earlier this year. According to SK Hynix’s initial tests, its GDDR6-AiM (memory gas pedal) application can increase AI processing speed by 16x and reduce power consumption by 80%. This is much faster than Samsung’s modified MI100s, but we don’t know what tools SK hynix used for testing, so we can’t directly compare.
