First he found the characteristic equation for the homogeneous equation, [itex]D^2+ 3D+ 2= (D+ 1)(D+ 2)= 0[/itex] so that D= -1 and -2 and the "complementary solution" is [itex]C_1e^{-x}+ C_2e^{-2x}[/itex].

Since the right hand side is 20cos(2x), you look for a specific solution of the form Acos(2x)+ B sin(2x). Put that into the equation and solve for A and B.