I would consider tryptophan to be the most hydrophobic residue.
Hydrophobicity is not solely a measure of how non-polar a molecule is, but
also its propensity to aggregate in aqueous solvent. The latter is directly
proportional to the accessible surface area of the molecule. Thus tryptophan
being both non-polar and having the largest accessible surface area would be
the most hydrophobic.
A scale of hydrophobicity was calculated by Nozaki and Tanford (1971) from
measurements of the free energy of transfer of amino acid side chains from
a non-polar solvent to water, and tryptophan was seen to be the least soluble.
On the same scale the least hydrophobic was arginine. This is essentially for
the same reasons : the molecule is small, with a small accessible surface area
and has several ionisable groups.

Nozaki and Tanford also measured hydrophilicity directly by partioning amino
acids between aqueous solvent and vapour (N-methylacetamide) phases. They found
the amino acid which partioned most into water was arginine (agrees with
hydrophobicity index). However, the least hydrophillic was not the most
hydrophobic, ie tryptophan, it was leucine.

This difference confirms the basic point about accessible surface area being
important. Hydrophobicity by definition also measures the tendency of molecules to aggregate in aqueous solvent.