For deterministic phase retrieval, the problem of insignificant axial intensity variations upon defocus of a smooth object wavefront is addressed. Our proposed solution is based on the use of a phase diffuser facilitating the formation of a partially-developed speckle field (i.e., a field with both scattered-wave and unperturbed-wave components). The smooth test wavefront impinges first on the phase diffuser producing the speckle field. Then two speckle patterns with different defocus are recorded at the output plane of a 4f-optical filtering setup with a spatial light modulator (SLM) in the common Fourier domain. The local variations of the recorded speckle patterns and the defocus distance approximate the axial intensity derivative which, in turn, is required to recover the wavefront phase via the transport of intensity equation (TIE). The SLM setup reduces the speckle recording time and the TIE allows direct (i.e., non-iterative) calculation of the phase. The pre-requisite partially-developed speckle field in our technique facilitates high image contrast and significant axial intensity variation. Wavefront reconstruction for the 3D refractive test object used demonstrates the effectiveness of the technique.