Evidence from monkey and human studies reveals two functionally distinct cortical streams in processing visual information: a dorsal attention pathway (‘where pathway’) and a ventral object perception pathway (‘what pathway’). Anatomically, the ventral pathway receives inputs mainly from the upper visual field (UVF), whereas visual information in the lower visual field (LVF) is largely projected to the dorsal pathway. We therefore hypothesized a functional asymmetry between LVF and UVF as the shape analysis was more efficient when stimuli presented in the UVF whereas attentional modulation was more prominent in the LVF. To test this hypothesis, we used fMRI to measure subjects' neural activity in the primary visual cortex retinotopically corresponding to Gabor patches presented in the UVF and LVF respectively while they judged the Vernier alignment between one pair of Gabor patches presented in a 45 or 135 degree titled axis and ignored the other pair orthogonal to the first one. Irrelevant to the task, a centrally-presented Gabor patch was either in line with or orthogonal to the Gabor patches to be judged. As expected, we found retinotopically corresponding regions of a Gabor patch in the primary visual cortex showed a higher response when the patch was either attended (versus ignored) or aligned with (versus orthogonal to) the center patch, suggesting that the primary visual cortex is involved in both attentional modulation and perceptual grouping. However, the attention effect was significantly larger when the patch was presented in the LVF than in the UVF, whereas the grouping effect showed an opposite pattern as the grouping effect was larger in the UVF. This asymmetry suggests that attention and grouping are indeed two separate functions, although they both manifest similar characteristics in processing visual information.