The neural representation of action observation involves brain regions that encode goal-related information, such as the anterior intraparietal sulcus (aIPS) as well as other regions that encode action kinematics, including inferior frontal cortex and superior temporal sulcus (STS). Little is known, however, of the role that visual perspective plays in the coding of actions. In this fMRI repetition suppression (RS) study, subjects (N = 11) watched video clips of a hand reaching to grasp objects from different perspectives and detected rare targets in which the hand wore a ring. Novel or repeated grasp goal (utilize vs transport), perspective (first- vs third-person) and object type were presented in a 2×2×2 design. Surface-based analysis revealed that, in addition to typical action observation regions such as inferior frontal cortex, aIPS, and STS, a widely distributed network showed goal-, perspective-, and object-dependent RS effects. Repeated visual perspective on successive trials produced RS in left insula and cuneus, and right lateral occipitotemporal cortex, while goal repetition reduced the response in the right sylvian fissure. Regions previously implicated in action observation, however, appeared to code for interactions between goal, perspective and object repetition. RS to goal repetition varied depending on perspective in the right ventral premotor cortex and bilateral aIPS. In particular, it appeared that aIPS coded goal-perspective interactions, with either a novel goal or a novel perspective eliciting an increased response relative to repetitions of these stimuli. RS in left STS depended on repetition of goal and object, suggesting a coding of grasp kinematics in this region. In sum, we found that RS effects for goal-directed transitive actions depend on multiple factors including perspective, suggesting extraordinarily high-level encoding of actions. These results provide new details of the visual coding of actions within the human brain.