In the vocational education of Dutch as a Second Language (DSL) there is a growing need for oral proficiency tests that are efficient to use. In this article, we describe ways to make use of speech technology for rating speech quality. In two experiments, the rating of DSL-speech is compared to automated rating of several aspects of the acoustic signal. The correlations found between human and automatic ratings were relatively high. For that reason a third experiment has been set up using speech items that are representative of commonly used oral proficiency tests in DSL and that can be rated automatically by a Continuous Speech Recogniser system. The results obtained so far are discussed.