Also, to address your last paragraph, you shouldn't have to walk out. If you make it clear with the rest of the band you're at breaking point, they're likely to choose you over him and get rid of him, especially if you're putting the work in.