We study the geometry of non-reductive $4$-dimensional homogeneous
spaces. In particular, after describing their Levi-Civita connection
and curvature properties, we classify homogeneous Ricci solitons on
these spaces, proving the existence of shrinking, expanding and steady
examples. For all the non-trivial examples we find, the Ricci operator
is diagonalizable.