For low latency applications such as VOIP: load the VOIP core directly into PulseAudio and have a slim GUI frontend to control it.
There is currently a little bit of documentation available on how to write loadable modules for PulseAudio. Beyond this, read the source, Luke! If you are interested in writing new modules feel free to contact the authors in case you have any questions.