This work presents an approach for joint estimation of the pedestrian head and body orientation in the context of active pedestrian safety systems. It involves a probabilistic framework, where a set of orientation-specific detectors are used for each body part for both localization and orientation estimation, their responses being converted to a continuous orientation probability density function. To improve the localization, spatial anatomical constraints between the head and body are used, in a Pictorial Structure approach, to balance the part-based detector responses. The single-frame head and body orientations are integrated over time by particle filtering and estimated jointly to account for orientation restrictions and to obtain anatomical possible orientation configurations. The experimental evaluation is done over 65 pedestrian tracks in realistic traffic settings, obtained from an external stereo vision-based pedestrian detection system. The results show that the proposed joint probabilistic orientation estimation framework decreases the absolute mean head and body orientation error by approximately 15 degrees. Also, the system runs in near-real-time (8–9 Hz), which allows the use in the car.