Just In Time Architecture (JITA): A New Paradigm for Designing Agile Data Centers

By Cihan Tunc

Data infrastructures such as Google, Amazon, eBay, and E-Trade are powered by Data Centers (DCs) that contain tens to hundreds of thousands of computers and storage devices running complex software applications. Between 2013 to 2020, organizations’ investment in mobile, social, cloud, and big data technologies is expected to grow over 20 times faster than organizations’ investment in client/server technologies. Traditional IT architectures are not equipped to provide an agile infrastructure to keep up with the rapidly evolving application demands as they are distinct from the “traditional” enterprise applications, which means they have new and distinct system requirements. Common characteristics of next-generation applications are nonlinear scaling and relatively unpredictable growth as a function of inputs being processed. The dynamic changes in demand require the provisioning and re-provisioning of resources. Ultimately, this means considering new hardware designs that can enable distinct design principles for each unique workload.
Minimizing the inefficiencies in the deployment and management of a physical DC infrastructure requires a composable system that utilizes a set of flexible building blocks that can be dynamically and automatically assembled and re-assembled to meet changing workload needs. A composable infrastructure breaks down the resources into storage, compute, and network fabric resources that are treated as services. Control software logically assembles the hardware required by the application from the resource pool, eliminating the costly process of physically configuring hardware to support a specific software application. A composable infrastructure (also called software-defined infrastructure, Infrastructure as Code (IaC), decoupled infrastructure and hardware disaggregation) allows matching the resource requirements of a given application with the resources from its pool. The software developer defines the application’s requirements for physical infrastructure using policies and service profiles and then the software uses application programming interface (API) calls to create (compose) a Virtual Data Center (VDC) that can meet the application service level objectives. The physical location of the resources, used to build the VDC, is no longer a concern as they are treated as services. Shortened provisioning time and increased flexibility allow reducing both waste of resources and the time it takes to deploy a new application execution environment.
Industry has started to introduce systems that support a limited disaggregation capability. For example, the Synergy system by Hewlett Packard Enterprise and the Unified Computing System (UCS) M series servers by Cisco are two commercial examples of composable infrastructures. HPE Synergy allows the CPU to be decoupled from the storage and memory, but components remain physically close together because of latency. A fully composable system requires the ability to assemble individual processors and memory automatically with low overhead and latency. True disaggregation of individual DC components is only feasible with a high speed interconnect technology operating at Tera bit per second (Tbps) and low latency. It has been shown that for building DCs, by dynamically composing their basic components (processors, storage, memory devices, GPUs, special hardware), to be feasible from a performance perspective, their interconnection network must operate in >100 Tbps while the latency must be in the <10 microsecond range to support disaggregated data center infrastructures with a wide range of workloads.
In this project, we are investigating the architectural support required to implement the JITA architecture shown in the Figure. Requirements Analysis