Yes, you have the right to privacy. In the expectation of privacy, when you speak so another party, you have a right to expect that the conservation is private. Recording it without your knowledge violates that expectation.