Sometimes when I make a call using Sip Sorcery I get weird lag, and whats stranger if I restart the program by completely exiting and opening it again I sometimes get the same lag except I can still hear the end of the last call as well as the new call. Here is an example: https://www.youtube.com/watch?v=VZrfNwopgb0

This has happen on 2 separate computers and on various internet connections, my computers and internet shouldn't be a problem (i7 6700k, 16GB DDR4 RAM and so on).

That's what can happen with UDP. If you don't ask the other end of the call to stop sending audio (using a BYE request) it could keep sending for quite a while. Eventually it should time out when it doesn't receive any audio back or it gets an ICMP error message.

In this case where you restart the process within a short space of time the audio from the first call is still being sent and your second call assumes it's the new audio. Ideally you should send the BYE request to end the first call before ending the process. You could also use a random RTP port range to mitigate the issue, the approach I use is to use a set range which starts at the same point every time the application starts.