Smarts, for sure — but what other qualities make
a good lawyer?

New research proposes assessing the judgment and people skills
of law-school applicants — something the LSAT alone can't
do

By Carol Ness, Public Affairs|
4 August 2009

BERKELEY — The LSAT is,
in tandem with an applicant's GPA, the gold standard for
U.S. law-school admissions. While the test may do a great
job identifying potentially stellar law students, picking
the ones who will ultimately make the best lawyers takes a broader
approach, according to groundbreaking research by two Berkeley
experts.

Marjorie
Shultz

Marjorie Shultz, a Berkeley professor of law, and Sheldon Zedeck,
an industrial-psychology expert (and Berkeley's vice provost
for academic affairs and faculty welfare), have designed a way
to do just that — according to tests of their research
on 1,100 graduates of Berkeley Law and Hastings College of the
Law in San Francisco.

Their Law School Admissions Project identified 26 factors that
make for effective lawyering, going beyond the sheer brain power assessed by
the Law School Admissions Test to cover personal and interpersonal
attributes such as integrity, empathy, situational judgment,
and communication and people skills. Then they devised tests
to measure those attributes, and validated the tests by trying
them out on the alumni, whose abilities as lawyers also were
rated by their peers and supervisors.

"Having correlated the subjects' test scores with their
ratings on those factors, we are able to say we can predict the
effectiveness of an attorney much better than the LSAT does," says
Zedeck.

The results of the work, if re-confirmed by broader testing,
could mean profound changes for law schools and the legal profession,
asserts Berkeley Law Dean Christopher Edley, who has supported
Zedeck and Shultz's work.

"I can't overstate the potential of this work to transform
the way law schools conduct selected admissions and, perhaps,
the way law firms think about training and evaluating young attorneys," says
Edley.

However, coming up with the funding to take the research to
the next level — a national study — has proven a
challenge, since its principal sponsor, the Law School Admission
Council (LSAC), has balked at going further, according to Edley
and the two researchers. LSAC, a nonprofit organization, owns
and administers the LSAT, in addition to marketing extensive
LSAT prep materials and admissions software for law schools.

The LSAC hasn't formally declined Zedeck and Shultz's request
for more funding, and when contacted by e-mail through a spokesperson,
council President Dan Bernstein didn't comment directly, instead
praising their work and talking about "the challenge" of
building on it. Asked to clarify, the spokesperson said the council "is
moving ahead on several fronts to see how noncognitive factors
might be of assistance to the law school admission process," but
she could not be more specific.

Whatever LSAC does, Zedeck and Shultz say their own talks with
the council over further testing have stalled for so long that
they believe the issue is moot. What's the holdup? For one thing,
says Zedeck, the council has expressed concern that their tests
might be coachable. His response: "You could ask the same
question about those who do well on the LSAT."

Edley comments, "Given the extraordinary quality of this
research, the only excuse I can imagine for LSAC refusing to
invest more is that they don't want to undermine the market power
of the LSAT. That's the problem with private-sector funding for
truth-telling research."

Dismaying realizations

The project got its start 11 years ago, as the number of minority
students entering Berkeley Law (then Boalt Hall) plunged after
Proposition 209, passed by voters in 1996, put an end to affirmative
action in the public arena in California. That meant an end to
the use of any racial preference in admissions at public schools,
including the University of California.

Sheldon
Zedeck (Peg Skorpinski photo)

Shultz and other law faculty were concerned. A committee appointed
in 1998 by Herma Hill Kay, then Berkeley's law dean, and headed
by law professor Malcolm Feeley analyzed the post-209 admissions
process and came to realize, with dismay, how much it depended
on standardized tests, according to Zedeck and Shultz's report
on their work.

That trend has only intensified with the advent of such highly
publicized law-school rankings as U.S. News and World Report's,
which lean heavily on LSAT scores, the professors wrote.

The committee was well aware, Shultz has said, of research
suggesting that the factors measured by standardized tests like
the LSAT are ones that whites and some Asians tend to do well
on, while African Americans and Latinos don't — but that
those aren't the only factors that make for a good lawyer.

"Everyone said it would be great if we could measure what
would make a great lawyer, but that it can't be done," Shultz
recalls. But Feeley knew of Zedeck and his expertise in employment-selection
and validation models, with a particular emphasis on reducing
adverse impact against minorities. (He helped design the federal
fair- employment standard that says a test for employment must
be related to the job.)

Zedeck went to work with Shultz, a Berkeley law professor
for 32 years before retiring in 2008. Now, some $600,000 and
15 boxes of paper records later, they've nailed down 26 factors — including
intellectual skills — that could be used to evaluate a
would-be attorney.

How to tell if a question's telling

A test of situational judgment might ask something like
this, Zedeck says: Say you're in charge of a Democratic
candidate's election committee, and it's scheduled to speak
to a Republican group about the candidate's stands on health,
crime, and taxes. Just before the event, the tax expert
on the committee is in an accident and can't make it. Do
you cancel the meeting, go but don't talk about taxes,
or go and talk about taxes?

The questions were tested on Berkeley and Hastings graduates
to determine which answers correlated with good lawyering. If
highly effective lawyers (as evaluated by peers and superiors)
picked the first answer to a given question, and bad lawyers
picked the third, the question was judged to be telling.

The 26 factors include but go beyond the skills tested by the
LSAT — analysis, reading and logic. "Your LSAT
score won't tell you whether you have integrity or diligence," Shultz
says.

In addition, while research indicates that standardized tests
that focus on academic skills put minority candidates at a substantial
disadvantage, Shultz says "there is not a race or ethnicity
difference in performance on our tests."

"That's why I say fairness demands that you have a system
that's a bit broader than the one currently used to pick applicants," she
adds.

Christopher
Edley, law dean

Since their report came out last fall, at least 18 law school
deans from around the country and some 40 law faculty have expressed
interest in the findings, Shultz says.

They've also heard from many lawyers, including Berkeley Law
alumni. Some were concerned that the work was tantamount to undermining
the school's effectiveness by lowering its standards, Shultz
says. She invited conversations that she says ended up moving
all the naysayers at least to neutral on the question.

"I wanted them to know that our tests are race-blind,
that we are not putting our thumbs on the scale," she says.

Even UCLA law professor Richard Sanders, one of the country's
best-known affirmative action skeptics (his work contends that
affirmative action makes it harder for African Americans to succeed
in law school), told the California Bar Journal that
he had no problems with Shultz and Zedeck's approach, though
it needed broader testing, the journal wrote last month.

Mostly, the response has been "wow, this would be really
great if it could actually work," Shultz says. "We've
gotten many emails from lawyers who say, 'I am your poster child,
I had terrible LSAT scores but I had this very important and
satisfying legal career.' "

The Bar Association of San Francisco, which recently launched
its own Destination Law School to help diverse students across
the college-to-law-school gap, is extremely interested in the
Berkeley research and invited Shultz to address several conferences.

"This is really, really important," says Yolanda
Jackson, BASF deputy executive director and diversity director. "A
lot of college students who have dreamed about being lawyers
get stopped at the LSAT point." Broader testing would reveal
the intelligences and life skills they do have, she says.

The Society of American Law Teachers and the Clinical Legal
Education Association have also shown an interest, according
to Shultz.

And some law firms have asked to use Shultz' and Zedeck's materials
in their own hiring decisions.

But for now, Shultz says, the researchers want to keep the
focus on law school admissions, and a national test of the findings, "because
that's where we are going to expand the pool of potential lawyers." Law
school is an avenue to significant amounts of money and influence,
and it's important that access to such a scarce and valuable
resource be fair, she adds.

A new phase of the project would try out Zedeck and Shultz's
tests on graduates from a wide range of law schools. If it finds
funding and turns out as well as the work so far, Edley says
Berkeley Law "definitely will participate in an experiment
with other leading law schools to test it in practice."