Go over 2.1-2.3 of [V] and list the results you are NOT familiar
with.

Tell me what you remember about the delta method. Should we review
this?

Jan. 10

Review: HJ

Continued with discussion of M and Z estimators, and started on
consistency (for M estimators).

References: 5.1 and 5.2 in [V]

Homework:

Consider the maximum likelihood estimator in the normal model with
unknown mean and variance. Write out the MLE for the unknown bivariate
parameter $\theta$ as an M-estimator and a Z-estimator, finding the
functions $m$ and $\psi$ explicitly.

Is a method of moments estimator an M-estimator, a Z-estimator, or
neither?

Jan. 12

Review: HJ

Theorems 5.7, 5.9 and Lemma 5.10 with proofs.

References: 5.2 in [V]

Homework:

Prove Theorem 5.9 in the textbook (loads of hints given in class).
Write out the details.

Use Theorem 5.7, Theorem 5.9, AND Lemma 5.10 to show that the MLE in
the normal case is consistent. It's OK if you get stuck on the uniform
convergence though, but check as many of the conditions of the theorems as
you can.

Suppose that we know ahead of time that the pararameter space is
compact for either the M or the Z estimator? How does this simplify the
issues you faced in the previous question?

Suppose that $X_n\rightarrow\theta$ in probability, and let $f$ be a
function which is continuous at $\theta.$ Prove that
$f(X_n)\rightarrow f(\theta)$ in probability.

Re-do the heurisitic justification of both the MLE and the
Z-estimator asymptotic normality when the unknown parameter is bivariate.
You will need to know the bivariate version of Taylor's theorem.

Jan. 26

Review: ZY

Theorem 5.21 with proof, Theorem 5.23

References: 5.3 of [V]

Homework: Questions 6,7,10,11,17,19 (using theory from
5.3, please).

Jan. 31

Review: AF

Two examples (mean and median) and proof of Theorem 5.23.

References: 5.3 of [V]

Homework:

Suppose that $X_i$ are IID from the Uniform distribution on $[0,
\theta].$ The MLE of $\theta$ is $X_{(n)},$ the maximum of the
observations. Show that the MLE is consistent and find the right scaling
$a_n$ to obtain a nontrivial limiting distribution for
$a_n(X_{(n)}-\theta).$ You should find that the limit is not Gaussian -
why not?

Read Example 5.28 (robust regression) and find conditions on the
function $\psi$ such that the Z-estimator is consistent and asymptotically
normal.

The idea behind the LASSO is not limited to statistics: look up L1
regularization. Where else is this used?

What does the acronym LASSO stand for? Why does this make sense?

Watch SHRINK
IT, read the lyrics (in the comments), and write at most one page
matching the lyrics to what you know about the LASSO - including what we
cover on March 2nd and 7th. Hand in your page on March 9th.

Here is a
cool real-world example of using the LASSO. I would
love to track down the original paper too.

March 2

YZ/YXZ continued their presentations. We started the discussion of
the penalty function.

References: For the next little while we will be
working from [SCAD paper].

Homework: From [ISLR] Section 6.8: questions
1,2,3(a),4(a),5(part d is for MA),7(careful version of a question I had
asked before),11

March 7

Review: NA

Role of the penalty function in bias/sparsity/continuity; SCAD; prep
for theoretical results (notation).