How does the open domain-specific architecture relate to chiplets and generative AI?

The Open Domain-Specific Architecture (ODSA) is a project within the Open Compute Project (OCP) community to establish open physical and logical die-to-die (D2D) interfaces for chiplets. The goal is to democratize the design and use of chiplets for domain-specific high-performance computing (HPC) applications like generative artificial intelligence (AI).

Domain-specific architectures (DSAs) are an emerging approach to designing and using heterogeneous chiplets. DSAs are structured to optimize data flows through hardware acceleration and support programming flexibility. In addition to generative AI, typical domains include blockchain, automotive, internet of Things (IoT), image processing, and so on. Improved energy efficiency is one of the key benefits of using DSAs (Figure 1).

Figure 1. Compared with general-purpose computing (top), DSAs (bottom) are optimized to boost performance, flexibility, and energy efficiency in specific applications. (Image: *McKinsey & Company*)

There are three general characteristics of a DSA:

High energy efficiency by using specialized and optimized processing. Specialized processors accelerate frequently used domain-specific computations in hardware like a machine learning core or a neural network (NN) chip for generative AI applications. High energy efficiency also contributes to improved thermal performance.
Programming flexibility for domain-specific operations. For example, DSAs designed for NN inference should be able to accommodate the processing needs of multilayer perceptrons, recurrent NNs, and convolutional NNs. These DSAs should be able to implement NN inference operations that are inefficient or difficult to run on general-purpose platforms.
DSAs must be able to integrate heterogeneous semiconductor IPs and multiple process nodes that are needed to achieve the dual goals of energy efficiency and programming flexibility.

ODSA workgroups
ODSA’s charter is to build on existing standards for connectivity and other functions and define an open interface and architecture that supports the use of chiplets from different vendors. The ODSA Work Group has developed new open IP and specifications in the application, memory management, link, and substrate layers (Figure 2). ODSA is structured with multiple subgroups, including:

The ODSA PHY Interface group is working to define a simple, open, flexible data-rate interface between chiplets. The group has defined a new bunch of wires (BoW) interface that combines simplicity of design, process portability, and compatibility with low-cost packaging technologies.
The ODSA Proof of Concept group is working to validate various technology proposals from the program in terms of the business and technical issues related to having multiple companies collaborating and sharing responsibility on a chiplet solution.
The ODSA Business Working group is tasked with defining a workflow and related business processes to enable companies to assemble commercial chiplets. It includes the Chiplet Design Exchange (CDX) subgroup that’s developing a chiplet design kit.

Figure 2. The ODSA Work Group has developed new open IP and specifications in the application, memory management, link, and substrate layers (Image: *ODSA Work Group*).

Chiplet design kit
The Chiplet Design Exchange (CDX) is an open-source subgroup operating in OCP’s ODSA project. The CDX working group includes experts with diverse knowledge, including electronic design automation (EDA), system design, IC and SiP design, outsourced semiconductor assembly and test (OSAT), IC fabrication, and material supply.

The group has developed a series of machine-readable models for the electrical and mechanical properties of chiplets, now embodied in JEDEC JEP30 PartModels. The models for the basis of a Chiplet Design Kit (CDK). JEP30 is a comprehensive document and covers the structure for the assembly process classification, electrical, physical, thermal, material deceleration, and supply chain, including product change notice and product discontinuance data.

Summary
The ODSA project aims to bring the benefits of domain-specific architectures to chiplets. It’s leveraging chiplet’s ability to support heterogeneous integration and bring performance improvements in terms of energy efficiency and flexible programming for domain-specific applications like generative AI. The ODSA effort is well underway way and has already resulted in the development of a Chiplet Design Kit.

References
Challenges and Opportunities to Enable Large-Scale Computing via Heterogeneous Chiplets, arVix
Domain-specific architectures and the future of compute, McKinsey & Company
Domain-Specific Architectures: Research Problems and Promising Approaches, ACM Transactions on Embedded Computing Systems
JEDEC and Open Compute Project Foundation Pave the Way for a New Era of Chiplet Innovation
The OCP Open Domain-Specific Architecture (ODSA) Subproject Makes Significant Gains in Chiplet-based Architecture, Design and Industry Collaboration
The Open Domain-Specific Architecture: A Chiplet-Based Open Architecture, ODSA Work Group
Which Data Interfaces Are Used for Chiplet Interconnects?, Cadence