VSCSE - Virtual School of Computational Science and Engineering

Sites

University of Illinois at Urbana-Champaign, National Center for Supercomputing Applications, Urbana, IL

Clemson University, Clemson, SC

Louisiana State University, Baton Rouge, LA

Marshall University, Huntington, WV

Michigan State University, East Lansing, MI

Princeton University, Princeton, NJ

Purdue University, West Lafayette, IN

University of California Los Angeles, Los Angeles, CA

University of Delaware, Newark, DE

University of Tennessee Knoxville, Knoxville, TN

University of Texas at Brownsville, Brownsville, TX

University of Texas at El Paso, El Paso, TX

University of Utah, Salt Lake City, UT

Proven Algorithmic Techniques for Many-core Processors

July 29  August 2, 2013

Studying many current GPU computing applications, we have learned that the limits of an application's scalability are often related to some combination of memory bandwidth saturation, memory contention, imbalanced data distribution, or data structure/algorithm interactions. Successful GPU application developers often adjust their data structures and problem formulation specifically for massive threading and executed their threads leveraging shared on-chip memory resources for bigger impact. We looked for patterns among those transformations, and here present the seven most common and crucial algorithm and data optimization techniques we discovered. Each can improve performance of applicable kernels by 2-10X in current processors while improving future scalability.

Prerequisites:

Experience working in a Unix environment

Experience developing and running scientific codes written in C or C++

Basic knowledge of CUDA (A short online course, Introduction to CUDA, is available to registered on-site students who need assistance in meeting this prerequisite)

Although not required, knowledge from "Programming Heterogeneous Parallel Computing Systems," offered July 9-13 this year is highly recommended.

Instructors:

Wen-Mei W. Hwu, professor of electrical and computer engineering and principal investigator of the CUDA Center of Excellence, University of Illinois at Urbana-Champaign

David Kirk, NVIDIA fellow

John Stratton, Ph.D. candidate in Electrical and Computer Engineering and author of the exercise solutions to "Programming Massively Parallel Processors - A Hands-on Approach"

Course outline:

Introduction

why problem formulation and algorithm design choices can have dramatic effect on performance