Your approach is the variation of parameters method ($y_p=\nu_1y_1+\nu_2y_2$, solve for $\nu_1'$ and $\nu_2'$ and integrate), which works for any inhomogeneous equation. Arturo's re-writing of $\cos(t)\cos(2t)$ as $\frac{1}{2}(\cos(3t) + \cos(t))$ allows one to use the much simpler method of undetermined coefficients.
–
Eric NitardyMar 1 '11 at 0:12

@Eric, ah, ok. Thanks! That could have saved some work...
–
t.b.Mar 1 '11 at 0:52

@Eric, if you want to fill in the details in a separate answer, feel free to do so. I'm not going to do it.
–
t.b.Mar 1 '11 at 1:07

There's a general way of doing these things. You solve the homogeneous equation $y'' + y = 0$, giving $C_1\cos(t) + C_2\sin(t)$ for constants $C_1$ and $C_2$, then add to it a single solution $y_p(t)$ to the inhomogeneous equation $y'' + y = \cos(t)\cos(2t) = {1 \over 2}\cos(3t) + {1 \over 2}\cos(t)$. The result will be the general solution to your differential equation.

To find $y_p(t)$, you try $y_p(t) = a_1\cos(3t) + a_2\sin(3t) + b_1t\cos(t) + b_2t\sin(t)$. You plug it in and solve for $a_1,a_2,b_1,$ and $b_2$. Normally you just try combinations of $\cos(3t), \sin(3t), \cos(t),$ and $\sin(t)$, but since the latter two solve the homogeneous equation you have to stick a $t$ in front. I will trust Theo Buehler is right and that $a_1 = -{1 \over 16}$, $b_2 = {1 \over 4}$ and $a_2 = b_1 = 0$. Thus your general solution will be
$$y(t) = C_1\cos(t) + C_2\sin(t) -{1 \over 16}\cos(3t) + {1 \over 4}t\sin(t)$$
(The ${5 \over 16}\cos(t)$ term gets absorbed into the solution to the homogeneous equation).