You need to set a white point brightness according to your work environment, so that monitor white is a close visual match to paper white. Your colleague needs to do the same. This is standard operating procedure for critical work.

Your perception is heavily influenced by the ambient conditions, right down to application interface, which is why it's impossible to give a fixed number. The general recommendation is often 120 cd/m², but that assumes "average" conditions, whatever that is.