Abstract

In contrast to flying flies, walking flies experience relatively strong rotational gaze shifts, even during overall straight phases of locomotion. These gaze shifts are caused by the walking apparatus and modulated by the stride frequency. Accordingly, even during straight walking phases, the retinal image flow is composed of both translational and rotational optic flow, which might affect spatial vision, as well as fixation behavior. We addressed this issue for an orientation task where walking blowflies approached a black vertical bar. The visual stimulus was stationary, or either the bar or the background moved horizontally. The stride-coupled gaze shifts of flies walking toward the bar had similar amplitudes under all visual conditions tested. This finding indicates that these shifts are an inherent feature of walking, which are not even compensated during a visual goal fixation task. By contrast, approaching flies showed a frequent stop-and-go behavior that was affected by the stimulus conditions. As sustained image rotations may impair distance estimation during walking, we propose a hypothesis that explains how rotation-independent translatory image flow containing distance information can be determined. The algorithm proposed works without requiring differentiation at the behavioral level of the rotational and translational flow components. By contrast, disentangling both has been proposed to be necessary during flight. By comparing the retinal velocities of the edges of the goal, its rotational image motion component can be removed. Consequently, the expansion velocity of the goal and, thus, its proximity can be extracted, irrespective of distance-independent stride-coupled rotational image shifts.

By contrast, walking blowflies experience relatively large periodic body rotations caused by the walking apparatus. When flies are forced to walk straight in a visually impoverished environment, the stride-coupled yaw rotations of head and body are very similar, whereas roll and pitch body rotations are compensated to some extent by counter-rotating the head. Hence, even straight-walking flies experience relatively large rotational yaw gaze shifts of approximately 4 deg at a stride frequency of 12 to 15 Hz (Kress and Egelhaaf, 2012). The corresponding retinal image flow might impede spatial information processing and, thus, object-directed behavior.

Similar to most previous studies on object fixation, we relied in our experiments on the natural affinity of flies for salient vertical objects, but we improved the techniques to allow for a more precise analysis of gaze behavior. We initially analyzed the fixation behavior toward a vertical bar in a stationary environment where the only retinal image displacements were self-induced. We then challenged the fly with external disturbances by moving either the bar or its background to assess the limitations of the fixation system. These external disturbances might lead to a conflict between goal fixation and the optomotor following responses to background motion, as has been tested previously on tethered flying Drosophila (Fox and Frye, 2014; Fox et al., 2014). We addressed three specific questions concerning walking blowflies – (1) are stride-coupled gaze shifts compensated during object-induced fixation tasks? (2) What are the functional implications of potential stride-coupled gaze shifts for the fixation performance? (3) What information about the distance to the goal is contained in the retinal input during object-induced approach behavior?

RESULTS

We monitored the gaze behavior of walking blowflies, Calliphora vicina (Robineau-Desvoidy 1830) by using two high-speed cameras,
while the flies approached a black vertical bar representing the ‘goal’. One wall of the walking arena consisted of a projection screen for the visual stimulus that was composed of the bar and a random checkerboard background pattern (Fig. 1A). When the bar and background were stationary (‘stationary condition’), the retinal image was only displaced by the self-motion of the fly. Oscillating either the bar (‘moving bar condition’) or the background (‘moving background condition’) allowed us to analyze the consequences of external disturbances on approach and gaze behavior. We determined the position and gaze orientation of the fly by automatic tracking of marker points attached to the fly (Fig. 1B) and reconstructed the position of the bar on the retina (Fig. 1C).

Schemes of the walking arena, marker positioning and yaw orientation calculation. (A) Box-shaped infrared-transparent acrylic walking arena (60×70×30 cm, length×width×height). Blowflies were recorded with high-speed cameras while approaching a black vertical bar that was projected onto the front arena wall (70×30 cm, width×height). The volume filmed covered an area of 26×21 cm in front of the projection screen. (B) Marker positioning (white dots) on the thorax (body) and the head of the blowflies tested. Magnified images from the top and side cameras were depicted. Scale bars: 1 cm. (C) Illustration of body (dark blue lines) and head (red lines) yaw orientation estimation. Yaw angles (Φ) were calculated from vector orientations based on the pixel coordinates of tracked marker points (see Kress and Egelhaaf, 2012). The orientations were estimated in relation to the horizontal axis of the image, which corresponded to the horizontal axis in the walking area. Positive values indicate leftward orientations in relation to the horizontal, negative values indicate rightward orientations in relation to the horizontal. Orange and light-blue lines depict the calculation of the left and right retinal edge positions, respectively. Note that the retinal edge positions are calculated with respect to the actual head yaw orientation.

Flies were able to approach the bar, even if it oscillated at large amplitudes of 10 cm and with speeds of 5 cm s−1. When the bar was moving, the approaching fly had to continually adjust its walking direction to reach it (Fig. 2A). Walking flies revealed a characteristic stop-and-go behavior when approaching the goal (Fig. 2B,C). During stop phases, gaze direction was kept relatively constant (Fig. 2B). At the end of stops, flies sometimes changed their gaze direction, which was accomplished by a body saccade (arrow 1, Fig. 2B; supplementary material Movie 1). Saccades were not exclusively performed at the end of stop phases, but also during continuous walking (arrow 2, Fig. 2B). The sample fly fixated the left edge of the bar in the frontal visual field before it stopped, but switched to fixate the right edge using a saccadic turn after the stop (Fig. 2E).

In contrast to what might be expected for object fixation behavior, the sample fly performed periodic gaze modulations around the yaw axis. The modulations appeared to be coupled to the stride cycle (Fig. 2B). No compensatory head yaw rotations occurred during object fixation. The head and body yaw orientations were aligned most of the time, even during body saccades. The average angular difference between head and body yaw orientation was 1.25±2.27 deg (mean ± s.d.). Although stride-coupled body and head rotations and sideways movements mainly fluctuated with the stride-frequency (Fig. 2B,D), the forward translation velocity revealed a component of twice that frequency (Fig. 2C).

During an approach, the retinal size of the bar increased (Fig. 2F) and the retinal positions of its edges diverged. Generally, an edge that was not fixated in a certain eye region drifted to more lateral regions during the approach (Fig. 2E). Moreover, some details of the retinal input parameters were affected by the stride-coupled gaze movements (Fig. 2B,G). The retinal velocities of the left and right edges of the bar, for instance, increasingly differed from each other the closer the fly got to the bar (Fig. 2G).

These observations obtained from one fly as an example are quantified in the following sections with respect to five functionally relevant aspects.

At which retinal position is the goal fixated?

We calculated the distance between the bar's center and the end point of the fly's approach to assess which parts of the bar were targeted by the flies. The end points were distributed along the entire width of the bar (Fig. 3A), except for the stationary condition, which had a slight bias toward the left edge of the distribution. This bias might be caused by slight asymmetries in the random background texture (see the lower part of the stimulus, Fig. 1A). When either the bar or the background was moving, the asymmetry in the distribution vanished.

In order to verify that the bar was fixated in the frontal visual field, we estimated the gaze direction in relation to the center of the bar by determining how far from the center the visual midline of the flies crossed the plane of the bar. Similar to the end point positions of approaches, the final gaze directions in relation to the bar's center were distributed along its whole width for all stimulus conditions apart from the stationary one. Here, the flies tended to orient toward the bar's left edge. The similarity between the distributions of the end points of approaches and the gaze directions for all conditions indicates that the gaze of the fly was directed toward the location of the bar that they approached (Fig. 3A).

We determined the retinal position of the edges and of the center of the bar for each walk to assess whether walking flies stabilize the bar in a certain eye region. Flies sometimes changed the fixated edge during the early phase of their approach (Fig. 2; supplementary material Movie 1). Thus, we analyzed the retinal position only during the last 500 ms of the approaches. Most flies did not change their fixation preference during this period. Because the fixation preference for a certain section of the bar varied between walks, we
subdivided them into three classes, depending on whether the flies tended to fixate on the left edge, the center of the bar or the right edge. The classification was based on the criterion that the bar section with the lowest average retinal velocity was assumed to be the section that was fixated. On this basis, we identified 38 left-edge walks, 10 center walks and 43 right-edge walks. Of all the walks, 12 could not be classified unambiguously.

Individual walk toward the moving bar. (A) Approach trajectory to the black bar. The head positions (dots) and yaw orientations (lines) measured every 100 ms of the approach are plotted. Dot and line color indicate the position of the bar center. Warm colors code a position left of its central position in the arena, cold colors code a right position of its central position in the arena. Greenish colors code a central position in the arena. The black bar represents the bar position at the end of the approach. The black dashed line illustrates the textured background. (B) Head (red) and body (blue) yaw orientation in relation to a horizontal axis in the walk area. Positive values indicate a leftward orientation, whereas negative values indicate a rightward orientation in relation to a horizontal axis in the walk area. Black arrows indicate saccades after a stop phase (arrow 1) and during continuous walking (arrow 2). Difference angles between body and head yaw orientation are indicated by the green line. (C) Forward head velocity (walking speed) of the approaching fly. (D) Lateral head velocity of the approaching fly. (E) The azimuthal position of the bar in relation to the head yaw orientation of the fly (left edge, orange line; right edge, light-blue line; bar center, purple line). The dotted lines represent the movement direction of the bar: a positive slope stands for leftward motion, whereas a negative slope represents rightward motion. (F) Angular horizontal extent of the bar in the field of view of the fly. (G) Horizontal retinal edge velocities (left edge, orange line; right edge, light-blue line). In B–G, the vertical gray lines depict the touch down time of the left mid leg and, thus, represent the stride cycle timing.

Irrespective of whether the center or one of the edges of the bar was approached, the distributions of the retinal bar position were relatively broad (Fig. 3B). When the bar was stationary, flies predominantly fixated the respective bar section in the binocular eye region. The distributions of the retinal positions were more displaced with respect to each other for the moving bar condition (Fig. 3B). This characteristic was presumably a consequence of the more curved approaches that were necessary to reach the bar under this condition (Fig. 2; supplementary material Movie 1). When the background moved, retinal fixation positions were also slightly displaced compared with those under the stationary condition, but not as much as for the moving bar.

Directedness of approaches

In order to assess whether object or background motion affected how directly the goal was approached, we estimated the ratio between the length from the actual trajectory to the goal and the shortest possible trajectory (translation D-index=real translation/optimal translation; Fig. 4A) and the ratio between the sum of actual rotations and the minimal rotation that is required to orient toward the bar (rotation D-index=real rotation/optimal rotation). A D-index value of 1 indicates optimal behavior, and suboptimal behavior yields a D-index >1 (an example calculation is given in the caption of Fig. 4).

If the bar was stationary, a straight translation toward the bar would be optimal, whereas a slightly curved trajectory would be optimal if the bar was moving. Then, the calculated optimal trace was determined as a sequence of translations directed to the center of the bar. For each point in time of an approach (5 ms intervals), we calculated a translation step towards the center of the bar. Because the position of the bar center changed over time, the optimal approach trajectory had a curved shape. Optimal rotation behavior corresponds to the minimum rotation required to reach the center of the bar from the starting orientation. To obtain this optimal rotation angle, we summated all yaw angle changes that were required if the fly oriented along the calculated optimal trajectory during its approach to the bar.

Walking flies approached the bar on a near-optimal path with respect to translation, even if the bar or background were moving
(Fig. 4B). However, the overall rotations were four to five times larger than necessary for approaching the bar (Fig. 4C). These large overall rotations for all stimulus conditions are the consequence of the stride-coupled yaw rotations (Fig. 2B). The moving bar condition led to a significantly lower rotational D-index than when the bar and background were stationary or when the background moved. This difference is a consequence of the considerably larger optimal rotation values for the moving bar condition due to the movements of the bar because these values served as a denominator in the D-index calculation (median of summated real rotations: all stationary, 329 deg; moving bar, 302 deg; moving background, 366 deg; P=0.1, Friedman test).

Relative gaze direction and retinal position of the bar. (A) Final approach position and gaze direction in relation to the center of the bar. Gaze direction was assessed by the distance from the center of the bar where the visual midline of the fly crossed the plane of the bar to compare approach position and relative gaze direction. The bar center is indicated at the position of 0 mm. The edges were at −29 mm (left edge) and 29 mm (right edge). The probability densities were normalized with regard to the total amount of approaches for the respective conditions. Sample size: stationary bar and background, 34 walks; moving bar, 35 walks; moving background, 34 walks. (B) Probability density of retinal positions of the approached bar part during the last 500 ms of the approach phase. Following the convention of Fig. 2E, positive azimuthal angles represent the left visual field and negative ones the right visual field. The bilateral field of view ranges from approximately −10 to +10 deg. Sample sizes: stationary bar and background – left edge: 13 walks, bar center: 4 walks, right edge: 10 walks; moving bar – left edge: 13 walks, center: 2 walks, right edge: 16 walks; moving background – left edge: 12 walks, center: 4 walks, right edge: 17 walks.

Stop phases

Blowflies revealed a ‘stop-and-go’ behavior when approaching their goal (Fig. 2B). We defined stop phases as those time intervals with position changes between consecutive frames of less than 0.05 mm. Stop behavior varied between the conditions tested. A moving background caused significantly more stops than a moving bar (Fig. 5A), but was not significantly different from the stationary condition. The stop durations were similar under all conditions (Fig. 5B). Stop phases occurred at significantly larger distances to the bar for the moving bar condition than for the other two conditions (Fig. 5C). When the bar was stationary, the flies stopped at distances where the bar had, on average, a horizontal angular extent of 40 deg and a horizontal expansion velocity of 51 deg s−1 (Fig. 7A, Fig. 9A). Under the moving bar condition, flies preferably stopped at distances where the bar had a horizontal extent of 33 deg, when its lateral movement speed amounted to 27 deg s−1 and the expansion velocity to 28 deg s−1. The overall retinal movements of bar and background during the stop phases ranged between 6 and 7 deg.

Saccade-like changes in yaw orientation were especially prominent before and after stop phases and appeared to follow a stereotypic velocity profile. We defined a change in yaw orientation as a saccade when the yaw velocity exceeded 300 deg s−1 for more than 15 ms. On this basis, even small saccades could be detected while omitting the stride-coupled yaw changes. The saccades before and after stops reached amplitudes of up to 30 deg (Fig. 6A) and peak velocities of about 1000 deg s−1 (Fig. 6B). Although saccades were scattered along the entire trajectory before stops, they were aggregated in time at around 50 ms after the end of the stop phase. Approximately 47% of all stops were followed by a saccade with an average amplitude of 16 deg, independent of the condition tested (P=0.57; Friedman test). In addition, flies made at least one saccade with an average amplitude of 13 deg, independent of the condition (P=0.13; Friedman test), in 67% of walking phases before and after stops. The number of stops and the number of saccades during individual walks phases were negatively correlated (all stationary, R=−0.65; moving bar, R=−0.66; moving background, R=−0.86).

In order to test whether the saccades following a stop phase served to orient the fly toward the bar, we again segregated the walks into right- and left-edge approaches and calculated the distributions of the retinal position of the left or right edge of the bar before and 85 ms after stops. We analyzed all stops, only the first or the last stop, only stops followed by yaw saccades, and stops performed within the last second before reaching the bar. We did not find systematic shifts of the distribution of the fixated edge after the stop toward a more frontal position in the visual field for any of these alternatives (Fig. 6C, only ‘all stops’ data are shown). Hence, in contrast to the impression obtained from individual walks (Fig. 2A), stops did not serve to reorient toward the bar.

Do stops affect the directedness of the approach? In order to address this question, we compared the number of stops and the D-indices calculated for each individual fly. Translational D-Index values tended to increase with increasing stop numbers, indicating
less-directed approaches, although the correlations are relatively weak (all stationary, R=0.1; moving bar, R=0.16; moving background, R=0.51). This was different for rotations, here, increasing stop numbers resulted in lower rotational D-indices, indicating a more directed orientation (all stationary, R=−0.4; moving bar, R=−0.32; moving background, R=−0.52). Hence, frequent stops slightly impaired the translational directedness of the approaching flies and improved the rotational directedness, especially under the moving background condition.

A measure for approach directedness. The D-Index represents the ratio real/optimal of performed translations and rotations. (A) Example trajectories illustrating real and optimal approach translations. The translational D-index for the middle trajectory was 211.8 mm/192.9 mm=1.1. D-index for (B) translations and (C) rotations. D-indices were individually calculated for each fly by averaging over the three walks per visual condition and then averaged across all 12 flies per condition. Box and whisker plots represent the median (red lines) and corresponding quartiles (25th and 75th percentiles) with outliers (red plus). Translational D-indices did not differ significantly (P=0.17; Friedman test). Rotational D-indices were significantly higher in the case of the bar being stationary (irrespective of the background condition) compared with those when the bar was moving (*P<0.001, Friedman test).

Stride-coupled translations and rotations

The translational and rotational walking velocities of blowflies contained strong periodic components coupled to their stride cycle. Stride cycles were defined by the touchdown of the left mid leg (Fig. 2B–G). They lasted for 85±3 ms (mean ± s.d.) under all conditions (P=0.36; paired Student's t-test). In order to estimate the coupling between the stride cycle and the head and body
movements, we averaged the respective translational and rotational data within a time window of 40 ms before and after a stride cycle. We only took uninterrupted walking phases into account. Owing to the fact that stride-coupled translations and rotations varied only slightly between conditions (translations: forward, P=0.49; sideward, P=0.23; rotations: body, P=0.66; head, P=0.84; trace tangent, P=0.79; Friedman test), we averaged the data over all conditions.

Stop phase parameter across visual conditions. Stop data were individually calculated for each fly by averaging the three walks per visual condition and all 12 flies per condition. Box and whisker plots represent the median (red lines) and corresponding quartiles (25th and 75th percentiles) with outliers (red plus). (A) A moving background caused significantly more stops than a moving bar (*P=0.024; Friedman test). (B) Stop durations were similar under all conditions (*P=0.34; Friedman test). (C) Stop phases occurred at significantly larger distances to the bar for the moving bar condition than for the other two conditions (*P<0.01; Friedman test).

Walking blowflies moved forward by 7.9±0.9mm (mean ± s.d.) per stride cycle. The forward velocity was 95±13 mm s−1 and systematically modulated by 17% at twice the stride frequency (Fig. 7A). These modulations presumably resulted from two opposite leg triplet steps during a stride of the tripod gait. Sideward head translations oscillated by 0.7±0.4 mm at stride frequency (Fig. 7B) with an average lateral velocity of 6.6±0.2 mm s−1.

We related the stride-coupled body and head orientations to the orientation of the walking trajectory by taking the position of the central marker point on the prothorax in consecutive video frames and calculating the corresponding tangential orientations (note: this is not the body orientation, which may differ from the tangential orientation of the trajectory, especially when the fly generates movements with a sideways component; see Fig. 1C). The modulations of the orientation of the walking trajectory were much larger than those of the head and body (Fig. 7C). The large
trajectory modulations most probably resulted from lateral movements of the body during a stride cycle. From the phase relations between the stride-coupled rotations of the trajectory and the head and body, respectively, the body and head appeared to have counteracted the orientation of the walking trace. Thereby, the head followed the body with an inconspicuous time lag of 5 ms (cross correlation, Rmax=0.97; Fig. 7C). Moreover, the average stride-coupled rotational velocities of the body and head were considerably slower than the orientation changes of the walking trace (Fig. 7D). The stride-coupled angular velocity of the head and, thus, of the retinal image amounted to 107±4.21 deg s−1 (mean ± s.d.).

Yaw rotation and retinal bar position around stops. (A) Yaw rotations and (B) rotation velocities 125 ms before and after stop phases. The start of a stop phase (i.e. translation below 0.05 mm frame−1 in three consecutive frames) is set to 0 ms on the x-axis for the plot parts illustrating the period before the stops. In plot parts representing the period after the stop phase, the 0 ms mark represents the point in time when walking is reinitiated. We normalized the yaw angles presented to the orientation during stops to make the diverse orientation data comparable. Stops preceding or following the analyzed time window of 125 ms were excluded. Analyzed stops: all stationary, N=67; moving bar, N=37; moving background, N=89. According to our saccade definition, 40% of the illustrated stops were followed by a saccade, irrespective of the stimulus condition. (C) Normalized retinal position of the fixated bar edge before (0 ms in A and B) and at 85 ms after the stops. Walks were classified depending on whether the left or the right edge was fixated, as in the retinal bar position analysis (Fig. 3B). Following the convention of Fig. 2E, positive azimuthal angles represent the left visual field, negative ones represent the right visual field. The binocular field of view ranges from approximately −10 deg to +10 deg. Bar center approaches were not included in this analysis, as the sample size was comparatively low with two to four walks and six to 17 stops per condition. Sample size: stationary bar and background – left edge: 12 walks and 38 stops, right edge: 12 walks and 46 stops; moving bar – left edge: 13 walks and 25 stops, right edge: 16 walks and 20 stops; moving background – left edge: 12 walks and 47 stops, right edge, 18 walks and 79 stops.

Retinal image expansion while approaching the goal

Can walking flies make use of translatory optic flow to obtain information about the distance to the bar? In contrast to the intersaccadic intervals during flight, the omnipresent stride-coupled rotational optic flow may impair exploitation of spatial information.

Forward translation of the fly induced an expansion of the bar's retinal image. Expansion was proportional to the walking speed and inversely proportional to the distance of the bar when it was approached orthogonally. However, a stride-coupled rotational component was superimposed on the retinal expansion velocity, which led to retinal movements of the bar's two edges with the same speed and direction. By contrast, image expansion caused edge movements in opposite directions and at different velocities, depending on the approach angle of the fly and its distance to the edge (Fig. 8). Hence, the rotational velocity component is eliminated by subtracting the velocities of the right and left edges from each other, and the retinal expansion velocity can be extracted, and thus, nearness (i.e. the inverse of distance) information about the bar can be derived (Fig. 8).

We determined the retinal expansion velocity of the bar as a function of its nearness for all walks with translation velocities in the range of 8 to 14 cm s−1 and the bar center seen within ±45 deg relative to the frontal midline of the visual field of the fly. Irrespective of whether the bar or the background was moving, the expansion velocity experienced increased linearly with increasing nearness (Fig. 9A). The slightly larger expansion velocity at the end of the approach under the moving background condition might be a consequence of the more central approaches (Fig. 3A). The expansion velocity exceeded the average stride-induced image velocity (Fig. 9) at nearness values below 0.025 mm−1, and thus, distances to the bar smaller than 40 mm.

Averaged translations and rotations within a stride cycle (80 ms). Colored lines represent the averaged head data across conditions. Shaded areas represent the s.d. between conditions. The vertical line at the 0 ms mark indicates the stride cycle start or end, i.e. in the case of all six legs performing a step. (A) Averaged forward translation velocity of the head within a stride cycle. The horizontal red line represents the average forward velocity during walking. (B) Averaged lateral velocity of the head within a stride cycle. Positive velocities indicate left shifts, negative ones indicate right shifts. Averaged steps for the analysis of stride-coupled translations: all stationary=451; moving bar=542; moving background=439; from 12 flies. (C) Averaged yaw angles of the head (red), the body (blue) and the trace tangent (green) within a stride cycle. Trace tangent: tangential orientation of the time-dependent position trace of the central marker point on the prothorax. Positive angles indicate an orientation to the left, whereas negative ones indicate an orientation to the right in relation to a horizontal axis in the walk area. The individual mean orientations were subtracted to allow averaging over different walks. (D) Averaged yaw rotation velocities of the head (red), the body (blue) and the trace tangent (green) within a stride cycle. Positive velocities indicate left turns, negative ones indicate right turns. Averaged steps for the analysis of stride-coupled rotations: all stationary=273 steps from seven flies; moving bar=333 steps from seven flies; moving background=338 steps from eight flies.

The expansion flow experienced did not increase monotonically, but was modulated in a stride-coupled manner (Fig. 8). These modulations became larger the nearer the fly got to the bar. In order to quantify these translation-based modulations of the image flow, we estimated their amplitudes within a time window between 40 ms before and after the reference point of a stride cycle and the corresponding nearness of the fly to the bar for the same walks, as analyzed for Fig. 9A. We omitted stride cycles immediately before and after a stop phase. Stride-induced modulations of expansion velocity were similar for all stimulus conditions tested (Fig. 9B) and rose with the increasing nearness to the bar up to a nearness of 0.03 mm−1, corresponding to a distance of 33 mm. Stride-induced modulations were scaled with the overall increase of expansion velocity and, irrespective of the nearness, amounted to 58% of the overall expansion velocity. Hence, the amplitude of stride-coupled optic flow modulations may be another cue of distance information in relation to a goal.

DISCUSSION

Walking flies orienting toward a goal experienced relatively strong rotational yaw gaze shifts of approximately 4 deg, even during straight phases of locomotion. These gaze shifts were modulated at a stride frequency of approximately 12 Hz. This situation differs greatly from flying flies, where rotations and translations are largely segregated by an active gaze strategy (Boeddeker et al., 2010; Braun et al., 2010; Egelhaaf et al., 2012; Schilstra and van Hateren, 1999; van Hateren and Schilstra, 1999). Because translations were superimposed with stride-induced rotational retinal image shifts at all times in walking flies, the acquisition of spatial information might be impaired. Consequently, flies are likely to use different computational strategies to obtain distance information during flight and walking. The translational flow velocity exceeded the rotational one and might be used by the fly for distance estimation only if flies were closer to the bar than 40 mm. On the basis of our findings, whether walking blowflies really make use of the retinal expansion velocity or the amplitude of its stride-coupled modulations cannot be assessed (Fig. 9). However, a previous study has indicated that translation-induced edge motion is used by walking Drosophila as a distance cue (Schuster et al., 2002).

Given that optomotor-following responses were found to counteract retinal wide-field motion (Götz, 1975; Hengstenberg, 1993), it was unexpected that the directedness of object-induced walks in our walking experiments did not deteriorate when either the object or the background was moved. Moreover, wide-field motion caused externally only slightly changed the retinal fixation position (Fig. 3B). Hence, the gain of the optomotor system may be reduced during fixation behavior, potentially in a manner similar to that which has been concluded for tethered flying Drosophila in a fixation paradigm (Fox et al., 2014) and free-flying male blowflies chasing after a moving female-like target (Trischler et al., 2010). Moreover, consistent with our results on walking blowflies, it could be shown that tethered flying Drosophila do not track moving objects by head movements, but rather follow the background motion. Nevertheless, they are able to approach the object (Fox and Frye, 2014).

Translation-induced retinal bar expansion speed. The retinal velocity of the right edge is given by the light-blue line, whereas the orange line represents the left-edge retinal velocity. The expansion speed of the bar can be obtained (black line) by subtraction of the absolute edge velocities. The velocity data shown here are taken from a walk obtained under the stationary condition.

Significance of stop phases during walks

Walking behavior differs from that of flight by its pronounced stop-and-go characteristic. The function of the stops is not yet fully understood. One functional aspect might be that external motion can be better detected during stops than during walking, where the visual system has to distinguish it from self-produced image motion (Gilbert, 1997). This might be one reason why the number of stops increased under conditions of a moving background compared with those of a moving bar.

Another functional aspect of stop phases might be that self-produced image motion impairs fixation behavior. This could be one possible explanation why flies that stopped more frequently performed fewer saccades during continuous walking. Accordingly, stops might serve reorientation toward the bar when reinitiating walking in order to support fixation behavior. Although we got the impression from individual walks that this hypothesis might be valid, quantitative analysis revealed that retinal target positions did not change systematically to more central or lateral positions after saccades (Fig. 6C).

Moreover, we found that flies that stopped more frequently approached the object less directly, whereas their rotational directedness improved. This tendency was most obvious under the moving background condition, in which no systematic reorientations were apparent after stop phases (Fig. 6C, right). However, it is not clear from this finding whether a less direct approach just contained more stops, or if more stops caused a less direct approach.

Additional experiments are necessary to elucidate the function of stops during the fixation behavior of walking flies. In such studies, only one high contrast edge should be presented, because flies tended to swap fixation between edges in our paradigm (Fig. 2; supplementary material Movie 1).

The finding that flies preferentially stopped at certain distance ranges depending on the stimulus conditions indicates that the retinal size of the bar and/or its retinal velocity might trigger a stop. Recent findings suggest that stops of Drosophila are elicited after experiencing back-to-front motion to avoid collisions with walking conspecifics (Zabala et al., 2012). The retinal velocities at which Drosophila stop walking are similar to those observed in the present study. However, we did not find a stop preference during back-to-front motion. Moreover, we observed frequent stops under the stationary condition, suggesting that stops may also have other functions than to prevent collisions with another object.

Potential consequences of stride-coupled gaze shifts

The visual system of walking blowflies is continually confronted with stride-induced gaze shifts of 4 deg and rotational velocities around 107 deg s−1 at frequencies around 12 Hz. The motion detection system of the blowfly is able to resolve image motion well in this dynamic range (Egelhaaf and Borst, 1989; Hausen, 1984).
Hence, stride-induced gaze shifts can be predicted to affect the responses of motion sensitive neurons. Strong responses of motion sensitive lobula plate tangential cells (LPTCs) (Borst et al., 2010; Egelhaaf, 2006; Hausen, 1984) to stride-coupled gaze shifts could be of functional relevance for the reasons outlined below.

Retinal image parameters during forward translation toward the bar. (A) Bar expansion velocity as a function of the nearness of the fly to the bar under the different stimulus conditions. Nearness is the inverse of distance: 1/distance (mm). Data taken for forward translation velocities ranging between 8 and 14 cm s−1 and in which the bar center lies within ±45 deg in relation to the frontal gaze direction of the fly. Bin size: 5 mm. Colored lines give the average expansion speed per distance bin. Shaded areas give the s.d. within a bin. Dashed line gives the averaged absolute stride-coupled rotation velocity of the head (data from Fig. 7D). Sample size – all stationary: 34 walks of 12 flies, 6870 data points; moving bar: 35 walks of 12 flies, 7025 data points; moving background: 34 walks of 12 flies, 7358 data points. (B) Amplitude of stride-induced modulations of expansion velocity as a function of the nearness of the fly to the bar for the three stimulus conditions. Data taken only during forward translation at velocities ranging between 8 and 14 cm s−1 and walks directed toward the bar center within ±45 deg. Bin size: 5 mm. Colored lines give the average peak-to-peak expansion speed value per distance bin. Dashed line gives the averaged absolute stride-coupled rotation velocity of the head (data from Fig. 7D). Shaded areas give the s.d. within a bin. Sample size – all stationary: 34 walks, 633 strides; moving bar: 35 walks, 609 strides; moving background: 34 walks, 699 strides; from 12 flies.

A hypothetical mechanism to alleviate the consequences of rotational optic flow on distance estimation involves an efference copy that generates a rotation-proportional output. This mechanism might originate in the leg control networks in the thoracic ganglia. The efference copy might modulate the responses of the optomotor system to stride-coupled rotations. Other possible mechanisms could involve reafferences of mechanosensory systems dependent on the stride cycle, such as the haltere system (Nalbach and Hengstenberg, 1993; Sandeman, 1980a; Sandeman, 1980b) and the prosternal organ (Horn, 1982; Sandeman, 1980a). Haltere reafferences reflect the yaw rotation velocity of the body and, therefore, could be used to modulate visual responses to yaw rotations in conjunction with reafferences of the prosternal organ, which detects the head orientation in relation to the body (Preuss and Hengstenberg, 1992). Although an efference copy and reafferences from mechanosensors might well be utilized to reduce the impact of self-induced rotations on visual information processing, both mechanisms cannot exactly predict the visual responses of LPTCs to self-rotations: LPTC responses do not only depend on stride-coupled retinal velocities, but also on the spatial frequency content and local contrast of the stimulus (Borst and Egelhaaf, 1989; Egelhaaf and Borst, 1989; Straw et al., 2008; Warzecha et al., 2000).

Even purely visual mechanisms can be conceived to extract spatial information from the behaviorally generated optic flow. A segregation of optic flow fields into their rotational and translational components can, at least in principle, be accomplished computationally for most realistic situations (Dahmen et al., 2001; Koenderink, 1986; Longuet-Higgins and Prazdny, 1980; Prazdny, 1980). However, such a computational strategy might be too intricate to be accomplished by a nervous system. Nonetheless, the problem might be solved in a computationally cheaper way for the special case of an object that needs to be approached. A motion-sensitive system might be able to cancel out the stride-coupled rotational optic flow and obtain image expansion velocities caused by translational self-motion by comparing the retinal velocities of the left and right edge of the bar throughout the approach. As the expansion velocity increases with the nearness to the goal, this information might be used for distance estimation. Moreover, not only the expansion velocity, but also the stride-coupled modulations increase with nearness to the goal and, thus, might be another potential source of distance information.

A hypothetical mechanism such as comparing retinal edge velocities appears to be biologically plausible, although an underlying neuronal correlate is unknown so far. Candidate neurons that might be involved are LPTCs, such as the figure detecting cells (FD cells) (Egelhaaf, 1985a; Egelhaaf, 1985b; Kimmerle and Egelhaaf, 2000; Liang et al., 2012) and the LPTCs, which have already been mentioned, that are most sensitive to wide-field motion. However, the most challenging demand for a network comparing edge velocities is the assignment of the corresponding edges to an object, especially in the presence of multiple objects and background structures in the environment. Such a task might profit from processes such as ‘selective attention’ (Pollack, 1988; van Swinderen, 2005; Wiederman and O'Carroll, 2013).

As indicated in Fig. 7C, walking flies rotated their head to the right, while they were translating to the left, and vice versa, throughout a stride cycle. As a result, the yaw rotation axis of the head and, thus, of the gaze is not located at its neck joint, but at a virtual (pivot) point in front of the fly. This situation is reminiscent of gaze behavior during orientation flights of hymenopterans called pivoting. Pivoting is thought to be one way to obtain relative spatial information while orienting toward a goal (Collett and Zeil, 1996). While pivoting, any object in the environment closer to the animal than the pivot point rotates in the opposite direction to the observer. By contrast, objects located at a larger distance than the pivot point rotate in the same direction as the observer. If walking flies employed such a strategy, they might be able to make use of the stride-coupled rotations to assess object proximity with respect to the pivot point. Taking into account the average yaw amplitude of 4 deg and the average sideways displacement of 0.7 mm, we estimated trigonometrically that the pivot point is located approximately 10 mm in front of the fly, which corresponds to one body length and, thus, to slightly more than the distance covered by one stride (8 mm). Given that walking flies are able to stop walking within one stride cycle [Calliphora (personal observation); Drosophila (Strauss and Heisenberg, 1990)], this relatively crude spatial information might be sufficient to avoid collisions during walking. Whether such a mechanism is relevant for walking flies needs to be tested in future studies.

Relevance for other studies on walking behavior

To the best of our knowledge, there is only one study in which the stride-coupled gaze shifts of freely walking flies while approaching a goal have been observed, although the implications for object-related behavior and spatial vision were not discussed (Horn and Mittag, 1980). The amplitude and frequency of stride-coupled gaze shifts were similar to those observed in the present study.

Another study on the head-body coordination of walking flies indicates periodic head and body yaw rotations of 5 to 10 Hz (Blaj and van Hateren, 2004). However, in contrast to our study, these rotations were interpreted to be saccades, similar to those performed during free flight (van Hateren and Schilstra, 1999). The head was reported to be stabilized around its yaw axis between saccades (Blaj and van Hateren, 2004). By contrast, we did not observe any walking phases in which the head was stabilized against stride-coupled yaw rotations (Fig. 2B, Fig. 7C) (see also Kress and Egelhaaf, 2012). Moreover, we observed that stride-coupled head rotations occurred on average 5 ms after body turns; i.e. the gaze was directed toward its previous direction slightly longer than the body (Fig. 7D). A possible reason why the conclusions of Blaj and van Hateren (Blaj and van Hateren, 2004) differ from ours might be that they attached coils to the head and body of the fly and that their flies were walking significantly slower than those in our experiments. The slower walking style in combination with the weight of the magnetic coils might have changed the gaze behavior of the flies. Moreover, their study did not provide stride data and, hence, was unable to assess how the stride cycle and the orientation of head and body might be coupled (for a more detailed discussion, see Kress and Egelhaaf, 2012).

Hence, we are confident that the stride-coupled gaze rotations observed here are genuine and that they are likely to have consequences for spatial vision, because walking flies do not generate relatively pure translational optic flow in contrast to flying ones. This conclusion is corroborated by a recent study that used novel leg and body tracking techniques to analyze gait characteristics in Drosophila (Mendes et al., 2013). Although this point is not explicitly made in this study, we can infer from scrutinizing the data that Drosophila also experience stride-coupled modulations of body translations and rotations similar to the ones that we describe here.

MATERIALS AND METHODS

Flies and preparation

We used female blowflies, Calliphora vicina, from our laboratory stock 1–3 days post eclosion. We briefly anesthetized the flies with CO2 and placed a drop of melted beeswax on each wing base to prevent them from flying.

We placed one marker point of nontoxic acrylic paint (Hobby Line, C. Kreul, Hallerndorf, Germany) on the head and three points on the thorax for the automatic tracking of head and body position and orientation (Fig. 1B) (Kress and Egelhaaf, 2012).

Experimental setup

We recorded the walking flies using two infrared-sensitive cameras (CR 600, Optronis GmbH, Kehl, Germany), equipped with DG MACRO 24–70 mm lenses (SIGMA GmbH, Roedermark, Germany) at 200 frames s−1. The walking arena consisted of an infrared-transparent acrylic box with a rear projection screen (Studio®, Gerriets GmbH, Umkirch, Germany) as the front wall. It was placed in a dark room (Fig. 1A). The left-side wall of the arena was covered with white cardboard containing a hole of a diameter of 7 cm for the side camera. The opposite side wall was equally textured, including a dummy camera hole to keep the arena appearance symmetrical. The arena floor was covered with black cardboard. The acrylic walls were specially coated to allow only light of wavelengths larger than 700 nm to pass through the walls (LUXACRYL-IR, TTV GmbH, Geretsried, Germany).

We used four custom-built LED panels as light sources. Two panels consisted of infrared (IR) LEDs with a peak emission of λ=890 nm, and the other two consisted of IR LEDs with a peak emission of λ=850 nm (GaAIAs Double Hetero, VISHAY Electronic GmbH, Selb, Germany). All panels emitted light at wavelengths far beyond the sensitive range of fly photoreceptors (Hardie, 1979). This led to an illuminance of 60 lx in the center of the recording area (recorded using an IL 1700 radiometer, International Light, Newburyport, USA).

The IR light sources and IR-sensitive cameras in combination with the specially coated acrylic walking arena guaranteed that the projection screen displaying the visual stimulus was the only perceivable light source for the flies tested.

Visual stimuli

We used an LCD projector with a refresh rate of 85 Hz and a resolution of 1280×720 pixels (PT-AX200E, Panasonic® Deutschland, Hamburg, Germany) for stimulus presentation. We ensured in preliminary experiments that this refresh rate was sufficient to elicit object fixation behavior, as established in earlier studies (Pick, 1976; Virsik and Reichardt, 1976).

Visual stimulation was controlled by custom-written scripts of the Psychophysics Toolbox Version 3 (PTB-3, http://psychtoolbox.org/HomePage) in MATLAB (The MathWorks, Natick, MA, USA). The visual stimuli consisted of a background pattern and a centrally positioned vertical black bar (‘stimulus’ in Fig. 1A). The background consisted of a random pattern of gray and white squares of 1 cm edge length and an angular extent of 2.9 deg at a distance of 20 cm. The Michelson contrast was 0.34 (luminance: gray squares, 32±3 cd m−2; white squares: 65±6 cd m−2). The bar was positioned in the center of the screen in front of the background. It had a size of 5.8×30 cm, corresponding to 16×56 deg at a distance of 20 cm. The bar (luminance: 1.4±0.2 cd m−2) had a Michelson contrast of 0.96 with the white squares of the background and of 0.91 with the gray squares of the background. Even at the initial distance of 20 cm, the bar and background elements already covered several neighboring ommatidia, given the interommatidial angle of 1.2 deg in the frontal visual field (Petrowitz et al., 2000).

Three types of stimulus condition were used. (1) Stationary condition: the bar and the background were stationary (‘stimulus’ in Fig.1A). Perceived stimulus motion solely resulted from self-movement of the fly. (2) Moving bar condition: the bar was oscillating around its central position in front of the stationary background at a constant velocity of 5 cm s−1 and a frequency of 0.25 Hz, resulting in movement of 5 cm to each side. (3) Moving background condition: while the vertical bar was stationary, the background was oscillating horizontally with the same movement parameters as the bar under the moving bar condition.

We installed two phototransistors above the walking arena facing the projection screen, which monitored the brightness changes resulting from the moving patterns to obtain positional information about the moving bar and the moving background. Each of the phototransistors was connected to an IR-LED, which signaled the pattern movement and was visible for the side camera. In this way, the timing of the texture movement could be monitored precisely.

Experimental procedure

Marked flies were released at the rear of the recording area while facing the projection screen. The synchronized cameras started recording and stored the images in a ring buffer (maximal recording time of 8 s). After the fly had reached the bar, we stopped recording and transferred the recorded walk to the computer.

We presented the different stimulus conditions in pseudorandom order and recorded three walks per condition for each of 12 flies. As the animals did not always walk toward the target after release into the arena, we defined a maximal trial duration of 2 min. If the fly had not reached the target during this time, we caught it and released it again at the rear of the recording area.

Video analysis

Fly position tracking and orientation calculations were conducted using the ‘2D method’, as previously explained (Kress and Egelhaaf, 2012). In short, the centroid of the white marker points on the head and body of the blowfly were automatically tracked frame by frame by the open source software ivTrace (http://opensource.cit-ec.de/projects/ivtools; Fig. 1B). Custom-written MATLAB scripts extracted the position and yaw orientation of the head and body from the marker coordinates. Orientation data were obtained from the vector orientation between the marker point coordinates (Fig. 1C) (Kress and Egelhaaf, 2012). Moreover, we estimated the retinal positions of the bar's edges from the positions of the bar and the fly's head orientation (Fig. 1C). The retinal position of the center of the bar was inferred from the bar edge positions.

Marker positions differed only minimally between animals. Hence, the estimated orientations of head and body yaw of different individuals might be afflicted with an offset. We corrected these systematic differences by adding the difference angle between calculated orientation and actual head and body orientation in a reference image. The corrected angles were within ±6 deg.

The 3D position of the head of the fly was determined by stereo triangulation of the head marker positions in the corresponding stereo camera images (Kress and Egelhaaf, 2012). We estimated the mean reconstruction error to be 0.3 mm (one-tenth of the head width). Stride cycle timing (McNeill, 2003) was manually registered by monitoring the touchdown time of the left mid leg.

ACKNOWLEDGEMENTS

We thank R. Kern and J. P. Lindemann for comments on the manuscript and K. Runte for his help in data acquisition. In addition, we would like to thank our anonymous reviewers for their extremely helpful comments and suggestions, as well as P. Moeller-Reusch for editing the manuscript and P. Saunders for proof reading.

FOOTNOTES

Author contributions

D.K. designed the setup, executed the experiments and analyzed the data. D.K. and M.E. interpreted the data and wrote the paper.

Competing interests

The authors declare no competing financial interests.

Funding

The study was supported by the Human Frontier Science Program (HFSP) and Deutsche Forschungsgemeinschaft (DFG).

(1985a). On the neuronal basis of figure-ground discrimination by relative motion in the visual system of the fly. I. Behavioural constraints imposed on the neuronal network and the role of the optomotor system. Biol. Cybern.52, 123-140.

(1985b). On the neuronal basis of figure-ground discrimination by relative motion in the visual system of the fly. II. Figure-detection cells, a new class of visual interneurones. Biol. Cybern.52, 195-209.

Similar articles

Other journals from The Company of Biologists

Many organizations that use sonar for underwater exploration gradually increase the volume of the noise to avoid startling whales and dolphins, but a new Research Article from Paul Wensveen and colleagues reveals that some humpback whales do not take advantage of the gradual warning to steer clear.

Many animals stabilize their vision by swivelling their eyes to prevent the image from smearing as they move. A new Research Article on tadpoles from Céline Gravot and colleagues shows that contrast between objects in their view affects the strength of this visual reflex, suggesting that the eye may be processing the image at a basic level to produce the reflex.

When starting her own lab at James Cook University, Australia, Jodie Rummer applied for a Travelling Fellowship from JEB to gather data on oxygen consumption rates of coral reef fishes at the Northern Great Barrier Reef. A few years later, Björn Illing, from the Institute for Hydrobiology and Fisheries Science, Germany, followed in Jodie’s footsteps and used a JEB Travelling Fellowship to visit Jodie’s lab. There, he studied the effects of temperature on the survival of larval cinnamon clownfish. Jodie and Björn’s collaboration was so successful that they have written a collaborative paper, and Björn has now returned to continue his research as a post-doc in Jodie’s Lab. Read their story here.

Where could your research take you? The deadline to apply for the current round of Travelling Fellowships is 30 Nov 2017. Apply now!