If you need the extra capability both off-road driving performance and space (and i mean boatloads) then LR4. The tech is ehhh, the reliability is tarred, and the dealerships nothing to write about.
If you need the on-road driving performance, in-car tech, and the ability to carry 4 adults and their gear then X5.

The lr4 really is phenomenal in capability but more of a long-distance family hauler for occasional use (and its prepared)
The x5 you can live with on a daily basis but you make a lot of compromises when making it a purposeful family hauler.

Good luck!

FYI:

X5d is great at towing, and really providing you with the necessary get up and go around the city.
X535i is great as a performer, but not great at towing
x550i is x5d but with a better highway performer.

Why does this matter?
CAUSE EVERY TIME someone says Lexus has better build quality, one should consider what Toyota their Lexus Engine and chassis platform comes out of and how long that part has been tested in the real world!