After adapting to a moving stimulus, the duration of another moving stimulus presented at the adapted location is underestimated (adaptation-induced time compression). On the other hand, the duration of moving stimulus is overestimated than that of static stimulus (time dilation). The compression effect has been reported to be selective to luminance motion and then considered to relate to early motion processing such as the magnocellular pathway. In contrast, the dilation effect has been considered to relate to higher motion processing such as the area MT, however, the luminance selectivity of this effect remains unknown. In this study, we directly compared the adaptation-induced time compression and the time dilation, in terms of luminance selectivity. In the experiments, we measured the time compression and the time dilation for luminance gratings and subjectively equiluminant color gratings (0.5 cpd, diam 8°). In the experiment of time dilation, observers compared the duration of a moving (7 Hz) standard stimulus and that of a static test. In the experiment of time compression, the adaptor was presented at the beginning of each trial, and observers compared the durations of two moving (7 Hz) stimuli: standard presented at the adapted location and test at another location. All stimuli were centered at 5° eccentricity. The standard duration was 600 ms and the test duration was variable (300-1200 ms). We found that the luminance motion induced significantly larger time compressions (~17%) than the equiluminance motion (~7%), consistent with the previous studies. Furthermore, significant time dilations were observed for the luminance motion (~22%), but not for the equiluminant motion. These results indicate the luminance motion induces larger motion-induced time distortions. Our study suggests that the early visual processing such as the magnocellular pathway is responsible for both the time compression and the time dilation.