Years of Citing Articles

Bookmark

OpenURL

Abstract

Programming distributed-memory machines requires careful placement of data to balance the computationalload among the nodes and minimize excess data movement between the nodes. Most current approaches to data placement require the programmer or compiler to place data initially and then possibly to move it explicitly during a computation. This paper describes a new, adaptive approach. It is implemented in the Adapt system, which takes an initial data placement, efficiently monitors how well it performs, and changes the placement whenever the monitoring indicates that a different placement would perform better. Adapt frees the programmer from having to specify data placements, and it can use run-time information to find better placements than compilers. Moreover, Adapt automatically supports a "variable block" placement, which is especially useful for applications with nearest-neighbor communication but an imbalanced workload. For applications in which the best data placement varies dyna...

Citations

...y---while also minimizing the number of messages---to minimize communication overhead and message delay. We assume that any node can reference any data element. We also assume the owner-computes rule =-=[8]. This mea-=-ns each data element has an &quot;owner&quot;, which is the only node that updates the element; however, other nodes may reference the element. The elements of a data structure can be placed on the no...

...hes determine data placements statically. They can generally be divided into two categories: using language primitives, such as the ones in HPF [9], or compiler analysis, such as the work reported in =-=[1]-=-, [5], and [12]. Language primitives involve the programmer in the choice of data placement; unfortunately, the best placement may be difficult or impossible for the programmer to determine. Compiler ...

...change when executing the program on a new architecture. With a compiler-based approach, the compiler infers a placement for each array in the source code by inspecting loops and array accesses (e.g. =-=[1, 12, 5, 2, 10, 14, 13, 19]-=-). Hence, the programmer need not be involved in placing data. However, a compiler may not be able to infer the best placement, especially for a dynamic computation. With a run-time system approach, s...

... compilers, or (less commonly) run-time systems. 1 This clustering is not contrived; it can occur in practice [6]. With language primitives, the programmer annotates each array with a placement (e.g. =-=[9, 7, 18, 21, 3]-=-). The advantage of using language primitives is that the programmer has full control over the program. However, the programmer might not know the best placement; even so, the best placement might cha...

...change when executing the program on a new architecture. With a compiler-based approach, the compiler infers a placement for each array in the source code by inspecting loops and array accesses (e.g. =-=[1, 12, 5, 2, 10, 14, 13, 19]-=-). Hence, the programmer need not be involved in placing data. However, a compiler may not be able to infer the best placement, especially for a dynamic computation. With a run-time system approach, s...

...change when executing the program on a new architecture. With a compiler-based approach, the compiler infers a placement for each array in the source code by inspecting loops and array accesses (e.g. =-=[1, 12, 5, 2, 10, 14, 13, 19]-=-). Hence, the programmer need not be involved in placing data. However, a compiler may not be able to infer the best placement, especially for a dynamic computation. With a run-time system approach, s...

...g workload and a placement-independent communication pattern. 3. Adapt and its Implementation The current Adapt prototype is implemented in concert with the Distributed Filaments (DF) software kernel =-=[4]-=-, which uses a DSM for communication. The Adapt system dynamically selects one of the data placements described in Section 2. It is given some initial data placement by the programmer or compiler (the...

... compilers, or (less commonly) run-time systems. 1 This clustering is not contrived; it can occur in practice [6]. With language primitives, the programmer annotates each array with a placement (e.g. =-=[9, 7, 18, 21, 3]-=-). The advantage of using language primitives is that the programmer has full control over the program. However, the programmer might not know the best placement; even so, the best placement might cha...

...data placements statically. They can generally be divided into two categories: using language primitives, such as the ones in HPF [9], or compiler analysis, such as the work reported in [1], [5], and =-=[12]-=-. Language primitives involve the programmer in the choice of data placement; unfortunately, the best placement may be difficult or impossible for the programmer to determine. Compiler analysis also m...

...mmer need not be involved in placing data. However, a compiler may not be able to infer the best placement, especially for a dynamic computation. With a run-time system approach, such as Adapt, ALEXI =-=[20]-=-, and CHAOS [11], data-placement decisions are made during execution. This approach can produce good placements for a larger class of applications because of the increased information available at run...

... involved in placing data. However, a compiler may not be able to infer the best placement, especially for a dynamic computation. With a run-time system approach, such as Adapt, ALEXI [20], and CHAOS =-=[11]-=-, data-placement decisions are made during execution. This approach can produce good placements for a larger class of applications because of the increased information available at run time, but it in...

...r compilers to make such decisions. Most current approaches determine data placements statically. They can generally be divided into two categories: using language primitives, such as the ones in HPF =-=[9]-=-, or compiler analysis, such as the work reported in [1], [5], and [12]. Language primitives involve the programmer in the choice of data placement; unfortunately, the best placement may be difficult ...

... the characteristics of the application change, it changes the placement again. The ability to change placements during execution is especially important for problems---such as particle-in-cell codes =-=[6]-=---- for which the best data placement can vary over the course of the application [17]. Adapt is currently implemented on a cluster of Sparc-1s and supports iterative scientific applications, which co...

...change when executing the program on a new architecture. With a compiler-based approach, the compiler infers a placement for each array in the source code by inspecting loops and array accesses (e.g. =-=[1, 12, 5, 2, 10, 14, 13, 19]-=-). Hence, the programmer need not be involved in placing data. However, a compiler may not be able to infer the best placement, especially for a dynamic computation. With a run-time system approach, s...

...ts for a larger class of applications because of the increased information available at run time, but it incurs additional overhead to do so. Other methods to remap data at run time have been studied =-=[16]-=-, but involve user intervention. We have presented an approach to data placement that allows the placement to adapt to the needs of the application. Adapt supports a larger class of problems than comp...

... In the Adapt CYCLIC program, these overheads are not present and the performance is very close to the DF program. 4.3. Particle Simulation Our particle simulation program models the behavior of MP3D =-=[15]-=-. We use a two-dimensional grid of space cells and parameterize the movement of particles to facilitate experimentation. Although our implementation simplifies the physics involved, the computational ...

...etermine data placements statically. They can generally be divided into two categories: using language primitives, such as the ones in HPF [9], or compiler analysis, such as the work reported in [1], =-=[5]-=-, and [12]. Language primitives involve the programmer in the choice of data placement; unfortunately, the best placement may be difficult or impossible for the programmer to determine. Compiler analy...

...bility to change placements during execution is especially important for problems---such as particle-in-cell codes [6]--- for which the best data placement can vary over the course of the application =-=[17]-=-. Adapt is currently implemented on a cluster of Sparc-1s and supports iterative scientific applications, which comprise a large subset of computational science applications. Performance on a network ...

... compilers, or (less commonly) run-time systems. 1 This clustering is not contrived; it can occur in practice [6]. With language primitives, the programmer annotates each array with a placement (e.g. =-=[9, 7, 18, 21, 3]-=-). The advantage of using language primitives is that the programmer has full control over the program. However, the programmer might not know the best placement; even so, the best placement might cha...

...change when executing the program on a new architecture. With a compiler-based approach, the compiler infers a placement for each array in the source code by inspecting loops and array accesses (e.g. =-=[1, 12, 5, 2, 10, 14, 13, 19]-=-). Hence, the programmer need not be involved in placing data. However, a compiler may not be able to infer the best placement, especially for a dynamic computation. With a run-time system approach, s...