Within a decade, the technological underpinnings for the process Gordon Moore described will likely come to an end as silicon photolithography approaches atomic scale. To continue the rapid, predictable, and affordable scaling of computing performance, major vendors have shown increased reliance on heterogeneous hardware acceleration, where multiple novel technologies are integrated into a single heterogeneous system, including different types of accelerators, new memory technologies and fast hybrid interconnect. Under such extreme heterogeneous environment, system design without carefully examining opportunities from various hardware and software stacks will no longer provide optimal efficiency. In this talk, I will showcase my most recent research on exploring holistic system design opportunities for two signature emerging workloads: highly interactive virtual reality processing for creating true immersive visual environment and deep non-linear convolution neural networks training. I will demonstrate how to apply holistic system design methods to address their individual challenge in today’s state-of-the-art design. In the end, I will discuss some potential future research directions for short and long term discovery.