How about : Speech Communication Made Simple: a multicultural approach by Paulette Dale and James C. Wolf.

I taught an ESL speech communcation class for 3 years. I used this book in an intensive ESL/academic prep program. My students were primarily Asian and were preparing for Masters work. However, the book's language is relatively simple and conversational and the art of crafting and delivering a speech is presented very plainly. I tried another ESL speech-making book that was well-done and a bit more complex in what it required from students, and decided to go back to this book even though I was teaching an advanced course. I think it would be good for your population of students too because it is easy to understand and interesting.

Speech making is easier to teach when you have more advanced students, but I've seen it be successful even with intermediate level students. They get a lot more fluent and confident as a result. I would just expect less complex speeches from them, although not necessarily shorter. They're always surprised when they churn out a 7 minute speech. They assume they can't speak that long but it just happens! I would also be less picky about grammar and pronunciation at that level, but I would still make it a point to encourage the students to take those lessons they're getting from other classes and apply them in their speeches. In the beginning, I would just focus on them getting comfortable before an audience and talking without stopping. Then, I would start to require that they learn to take a simple topic and organize it clearly and thoughtfully while giving good, specific details. Those are the basics that can be done at almost any level (except beginning). If you have more advanced students like mine were, you'd add more things like required research, debates, question and answer sessions, more accountability with pronunciation and grammar, more critical thinking, etc.. I think the book I recommended can really do good things across a variety of levels. Good luck!