Which Wisconsin circuit court judges are the most knowledgeable of the law, the most even-tempered and the fairest in the state?

And which are the least competent, most biased and quickest to anger?

In an exclusive, unique survey, USA TODAY NETWORK-Wisconsin asked practicing attorneys throughout the state to score the county judges they have appeared before. It is the second such survey, following the 2015 edition as part of our WisconINjustice series.

More than 4,000 responses produced ratings for 209 judges, or every circuit court judge for whom at least eight surveys were submitted.

Circuit court judges oversee criminal, civil and family law proceedings and routinely make decisions with lifelong consequences.

The USA TODAY NETWORK-Wisconsin Judicial Performance Database is one of few measures of these judges' effectiveness. It incorporates not just the attorney survey results, but also reports the number of times lawyers sought substitute judges to avoid their courtroom, and the number of times their rulings were overturned in appeals courts.

How we did it

In October, we reached out to nearly every attorney in Wisconsin to provide them with our survey. More than 1,000 responded, filling out more than 4,000 surveys.

That was enough to provide ratings for 209 of the state’s 249 circuit court judges, as we're publishing ratings only for judges with at least eight submitted surveys. The judges who were included had an average of 19 surveys submitted by lawyers.

Our survey was based on an evaluation program created by the American Bar Association. We then modified it for clarity, length and local relevance with input from professors of law and sociology at Marquette University and the University of Wisconsin.

We settled on 14 questions, each of which had a five-point scale. For each question — for example, “Is the judge prepared for court?” — attorneys could answer "extremely," "very," "somewhat," "a little" or "not at all." A 15th question asked whether the attorney would recommend that judge remain on the bench.

We then converted the answers into a five-point scale, so an answer of “not at all” was a 1 out of 5, and “extremely” was a 5 out of 5. The scores across the first 14 questions were averaged to determine the judge’s cumulative score from that attorney. The averages from all the responding attorneys were averaged to give a final cumulative score for the judge.

The average survey rating statewide was 3.8 out of 5, and 71 percent of surveys recommended retaining the judge who was being evaluated.

Judges' responses

Survey responses were emailed to each judge to allow a chance to respond, and for those who chose to do so we have included that with the survey, edited for length and relevance.

Our caveats

We took several steps to guard against duplicate voting — whether it be accidental or intentional — such as deleting multiple entries submitted from the same user.

Like any online poll, however, our survey was subject in part to the integrity of the people involved. We password-protected the survey to prevent general users from manipulating the data, but there is no way to guarantee attorneys didn’t find a method to cast multiple votes or that judges didn’t learn of the survey and evaluate themselves.

We also relied on the attorneys to follow the requirement that they evaluate only the judges whom they appeared before in the past year.

And some respondents in a survey of this magnitude will approach particular judges with something less than an even-handed assessment. Eighty-two surveys ranked the judge as a 1 out of 5 in every category.

Far more common were surveys that rated judges as a perfect 5 across the board. Those perfect surveys totaled 843, or about 21 percent of responses. There was no clear reason to doubt their validity, however, given the variety of times and places from which they were submitted.