John:
Perhaps you will find this reference useful:
Korn, E. L.,& Graubard, B. I. (1990). Simultaneous testing of regression coefficients with complex survey data: Use of Bonferroni t statistics. American Statistician, 270-276. Retrieved from http://www.jstor.org/stable/10.2307/2684345
The motivation for the proposal is the limited number of degrees of freedom for tests of multiple parameters in many surveys, where d.f. = no. sampled PSUs - no. strata.
Korn, E. L.,& Graubard, B. I. (1990). Simultaneous testing of regression coefficients with complex survey data: Use of Bonferroni t statistics. American Statistician, 270-276. Retrieved from http://www.jstor.org/stable/10.2307/2684345
On Nov 28, 2011, at 2:17 PM, Austin Nichols wrote:
John Antonakis<John.Antonakis@unil.ch>:
Mark and I promulgated simulation evidence in the 2007 UK meeting:
http://repec.org/usug2007/crse.pdf
that one coefficient with M clusters has a very good rejection rate
when you include fixed effects for clusters and use the cluster-robust
SE.
We also said
"Preliminary simulations show that the rejection rate rises from 5
percent to 100
percent as the number of coefficients increases from 1 to M."
and we fully intended to publish a paper right way. Life has
intervened, but there will be a paper (with a lot more new material)
one of these days.
Let me just interpret that finding in words, though. If you have 50
clusters and you are testing one coef, you have 49 df, if the
clustering produces a maximal loss of information. If you are testing
25 coefs, your rejection rate is not going to be anywhere near the
nominal rate, and if you are testing 51 coefs, you have a singular
variance matrix and the test will not work. Of course, a test of the
"model" with has three substantive coefficients and 49 dummies for
clusters will not fly, but each one of the 3 coefs should have a
pretty good SE and tests involving those 2 of those 3 coefs will only
slightly understate true variability of estimates.
But YMMV--run your own simulation on your data! (Imputing true
effects etc.) Or just cite the 2007 presentation. It reads as an
elliptical working paper, anyway.
On Mon, Nov 28, 2011 at 9:46 AM, John Antonakis<John.Antonakis@unil.ch> wrote:

Hi:
Awhile back, Mark Schaffer replied to a post, regarding a missing Wald test
when using the -robust- or -cluster(id)- option, noting:
http://stata.com/statalist/archive/2006-09/msg00840.html
Mark suggested that tests of individual parameters are still interpreted;
however, joint tests (for all parameters) should not (as I guess that the
test would not be trustworthy). If I remember correctly, he has mentioned
this in the past as have other posters too, directly or indirectly.
What I was hoping for is a published referenced the fact that parameter
tests are interpretable. Also, just wondering whether anyone has written
anything the consequences of this problem (i.e., insufficient clusters) on
tests of overidentification.
Best,
J.
--
__________________________________________
Prof. John Antonakis
Faculty of Business and Economics
Department of Organizational Behavior
University of Lausanne
Internef #618
CH-1015 Lausanne-Dorigny
Switzerland
Tel ++41 (0)21 692-3438
Fax ++41 (0)21 692-3305
http://www.hec.unil.ch/people/jantonakis
Associate Editor
The Leadership Quarterly
__________________________________________
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/