Advances in memory system architectures – Virtual Roundtable (part 2 of 2)

The second part of our Virtual Roundtable series on memories focuses on “advances in memory system architectures.” We are joined by Nando Basile (NB), Technology Marketing Manager for Memory with X-FAB, Bob Brennan (BB), Vice President of Emerging Memory at Micron.

JS: What design challenges should system architects be aware of when employing persistent memory?

BB: At the platform and rack level, system architects need to balance the triangle of compute, memory, and networking to optimize the application level bandwidth, latency, and capacity.

In the software infrastructure, the challenge is to make the new memory technologies appear to be seamless for mainstream usage while still providing the software architecture structures to optimize performance for the most demanding applications.

For any persistent media, security is a huge challenge. The system architecture (hardware + software) needs to protect data in motion, data at rest, and secure shredding of retired data. On this front, Micron is collaborating with the industry (ex. Trusted Computing Group, a consortium including AMD, HP, IBM, Microsoft, and others) and with customers to enable secure data storage solutions.

NB: This can be broken up into three main items. First, we have compatibility – so we need to ensure that the memory element supports the same operational parameters as the rest of the chip (but it must be understood that system architects will often be much more focused on the core chip features than the accompanying memory resource). Next, there is interfacing – where it is vital that the memory can work with the rest of the customer’s specific system, but without excessive IP investment being needed. Finally, putting an effective testing strategy in place is vital. The system architect will, for example, need to decide upfront whether to design for third-party testing, via an independent access to the memory block, or to invest in additional design resources – adapting their own proprietary architecture to access the memory and execute the test. Such decisions will usually have a major impact on the final chip cost.

JS: In which applications is persistent memory most useful? Where is persistent memory least beneficial?

NB: In embedded memory, there is no single ideal, universally-applicable solution. Finding the right technology to fit with an application will depend on a number of different factors – these include footprint, specific read/write activities, operating conditions, etc. Where do you see memory-centric computing or computational storage having the largest impact?

BB: Persistent memory will start with performance-focused applications like relational databases and NoSQL databases. As the technology further deploys to enable high-speed, high-capacity data storage close to the processor, and, as the support in the OS propagates to make it seamless, many applications will use the technology. As a historical comparison, NAND technology started as a performance tier and is now mainstream.

Persistent technology will be the least beneficial in applications which need consistent low nanosecond levels of performance or have a large amount of reads/writes, which require a lot of endurance.

JS: Where do you see memory-centric computing or computational storage having the largest impact?

BB: For more than three decades, the industry has discussed moving the compute close to the data instead of moving the data to the compute. Processing in-memory or near memory (PIM), and computational storage have thus far had limited commercial success. Acceleration computation is challenging; in order to compel a rewrite of existing software, there needs to be a 10X-100X performance and performance/power benefit. In the past, the data transition from the CPU to a device accelerator was difficult.

Today, platform architecture is shifting to enable coherent accelerators working in tight conjunction with the CPU cache and memory hierarchy, which will ease the software burden of moving, tracking, and optimizing data placement. In addition, new ML/AI applications have an insatiable and exponentially growing need for bandwidth, which will become very difficult to fulfill with classic compute and memory organizations. The combination of the new application demands and the new platform architecture will finally compel the deployment of new compute near memory and storage architectures. Once these performance-oriented applications have created the demand-pull to deploy these new architectures, they will waterfall to other application innovations we can’t yet imagine.

JS: What is the hottest topic in memory system architectures that designers need to be aware of?

NB: There is currently a lot of research into how to make use of part of a system’s embedded memory resource in order to ease the workload placed onto the CPU. This will be done inside the memory by executing basic computational operations in-place – thereby preventing some unnecessary back-and-forth data transfer. Such an approach would bring real benefits in terms of better power optimization, footprint reduction, and shorter computation time.

BB: We are at the tipping point of deploying a new memory hierarchy to put the right data at the right place and the right time. Leading hardware system architects are developing a platform, system, and rack solutions which provide new memory technologies tiered in a balanced pyramid of bandwidth, latency, and capacity. Leading software system architects are capitalizing on this hardware transition with new data placement algorithms to achieve higher levels of absolute performance and performance density than ever before. This virtuous spiral of hardware and software innovation on new memory hierarchies will set the backbone for innovation in end user experiences for the next decade.