When changing my avatar it doesn't seem possible to be able to remove the old one. The one from my profile is different from the most recent one I uploaded as seen here:
www.kunena.org/forum/profile/1230-josh
(you can see both on the same page)

The issue is that the directory /size144 is not updating the image while /size90 is. To further see this issue, when replying to this post you will see that the post preview avatar image is different than the one that displays on my profile.