Studies of reaction times for color discrimination have found faster responses to differences between (e.g. blue vs. green) compared to within (e.g. two shades of blue) color categories (e.g. Gilbert et al. PNAS 2006). The between-category advantage is more prominent in the right visual field and is abolished by verbal interference, consistent with an effect of language on the perceptual response to color. We asked whether an effect of linguistic category might be manifest early, in the perceptual encoding of color, by measuring the influence of color on perceptual grouping, a task which did not require a speeded response. Stimuli were composed of 5 1-deg circles forming the corners and center of a 4-deg square centered 8-deg in the left or right field. Diagonal corners had the same color and differed from the opposite pair by a fixed hue angle of 30 deg in CIELAB. Absolute angle varied over a range spanning blue and green. For each, the center color was varied in a staircase to estimate the angle at which the two diagonals appeared equally salient. Interleaved settings measured the angle of the blue-green boundary for the center spot presented alone. For corner colors spanning blue and green, a strong categorical effect predicts that the point of subjective equality (PSE) should remain tied to the boundary angle. Instead, PSEs varied monotonically with corner color angles, and did not consistently differ between the right and left fields. Perceptual salience as measured by grouping thus showed little influence of linguistic category. These results are consistent with other recent measures pointing to a lack of categorical effects on color similarity judgments (Lindsey and Brown JOV 2009), and suggest that the influence of language on color could occur late, e.g. at the stage of response selection.