They have their private key. They'll give you the public key to encrypt the data, which would need the private key to be decrypted.

You get a form of non-repudiation since the public key will decrypt data encrypted from their private key.

Thanks, I was filling out communication forms and they requested the public key. They are the ones to provide the key since my side knows the contents of the data. The receiver always has the private key.