This study is an analysis of one way that students in a language classroom may go off task, which is to start talking with other students about something not directly related to the current task or activity. The data come from video-recorded discussion tasks in English among Japanese university students. Multimodal conversation analysis is used to illustrate how the students use talk, gaze, and gesture to either curtail or sustain off-task interaction once it has been initiated. Focusing on instances of off-task interaction in which the students use English, the analysis illustrates how even students with fairly limited proficiency in English can show themselves to be interactionally competent while using English. The analysis also shows how off-task interaction may provide students with valuable opportunities for language practice and language learning.