Let $e_{i,(j)}$ be the residuals from regressing ${Y}$ onto all
${X}_{\cdot}$’s except ${X}_j$, and let $X_{i,(j)}$ be the
residuals from regressing ${X}_j$ onto all ${X}_{\cdot}$’s
except ${X}_j$, and let $X_{i,(j)}$.

If we regress $e_{i,(j)}$ against $X_{i,(j)}$, the coefficient is
exactly the same as in the original model.

Let $P_R$ denote projection onto the 1-dimensional model determined by the 1 vector.
Note that $P_F-P_R$ is again a projection: it projects onto the
orthogonal complement of the 1 vector within the $(p+1)$-dimensional full model. So, it
is a projection onto a $p$ dimensional space.

Now, let's take expectations. The first term is a constant, the cross term
has expected value zero and the expected value of the final term is
$$
p \cdot \sigma^2.
$$
This comes from the fact that
$$
\epsilon^T(P_F-P_R)\epsilon = \|(P_F-P_R)\epsilon\|^2 \sim \sigma^2 \chi^2_p.
$$

R will form these coefficients for each coefficient separately when using the confint function. These linear combinations are of the form
$$
a_{\tt lcavol} = (0,1,0,0,0,0,0,0)
$$
so that
$$
a_{\tt lcavol}^T\widehat{\beta} = \widehat{\beta}_1 = {\tt coef(prostate.lm)[2]}
$$

If we want, we can set the intercept term to 0. This allows us to construct confidence interval for, say, how much the lpsa score will change will increase if we change age by 2 years and svi by 0.5 units, leaving everything else unchanged.

Therefore, what we want is a confidence interval for 2 times the coefficient of age + 0.5 times the coefficient of lbph:
$$
2 \cdot \beta_{\tt age} + 0.5 \cdot \beta_{\tt svi}
$$

Most of the time, predict will do what you want so this
won't be used too often.

When comparing two models, one a special case of the other (i.e.
one nested in the other), we can test if the smaller
model (the special case) is roughly as good as the
larger model in describing our outcome. This is typically
tested using an F test based on comparing
the two models. The following function does this.