Monitoring of Referees Comes Under Scrutiny

Published: July 25, 2007

As David Stern concluded his news conference at a Manhattan hotel yesterday, discussing for 72 minutes what the N.B.A. knew about a suspected betting scheme involving one of its referees, he pledged to re-evaluate how referees are monitored on and off the court.

''What we do will be completely transparent,'' he said.

Yet in the four years that the N.B.A.'s current referee-evaluation system has operated, it has been anything but transparent, several N.B.A. referees said this week. The N.B.A. records every call by its referees, from personal fouls to three-second violations to double dribbles, as well as the instances when a potential call is missed. The league assigns one person to each of its 30 arenas, many of them former referees, to monitor the games in real time and cross-check the information afterward on videotape.

The data is collected and analyzed by the Segal Company, an actuarial consulting firm based in New York, which identifies rates of made and missed calls by officials. The findings are reviewed with referees by the league's executive vice president, Stu Jackson; its director of officials, Ronnie Nunn; four supervisors working under them; and the 30 game observers.

In an interview in April, Jackson cited three-second calls as an example of how the system works, saying the league could identify which referees were outside the norm in making that call.

''It may be because of his positioning on the floor that we identify by video,'' Jackson said. ''It may be because of a specific mechanic that he doesn't employ that we can correct and improve his accuracy.''

Stern said yesterday that the system existed primarily for referee development and not to uncover patterns that might suggest point-shaving.

''That is not to say that if something unusual popped up, we wouldn't pursue it,'' he said. ''But it was not predominantly developed as a screen for criminal activity.'' He added that the system detected nothing suspicious about Tim Donaghy's performance in recent seasons, and that in fact Donaghy was ranked ''in the top tier of accuracy'' when it came to blowing the whistle. Donaghy is being investigated for gambling on games and associating with organized crime figures.

Access to the evaluation system was clearly limited. Several N.B.A. referees said in interviews this week that they had been frustrated at not receiving data and other specific evidence to explain their tier ratings. They said that their repeated attempts to get hard numbers or see video of specific plays had been rebuffed by Jackson and those working under him.

''They came to me and said my call percentage was down, so I'm like, 'O.K., which calls did I miss?' '' said one veteran referee, who spoke on condition of anonymity because officials were prohibited to speak with the news media. ''And Stu said, 'We can't tell you that.' They won't tell you anything. They just throw a bunch of numbers at you and say, 'You need to get your percentages up or down.' It's just a big secret.''

N.B.A. referees are generally told whether they rate in the top third, middle third or bottom third of various categories, like missed traveling violations. These rankings are used for both hiring and firing decisions, regular-season game assignments and selection for playoff rounds, which are a considerable source of extra income for referees.

Another official said: ''I've asked Stu, 'For me to not know the stats, how can I improve? Show me the plays.' He'll say: 'Your three seconds are low. We don't know if you missed any, but you're below average.' Well then why do you say I missed them?

''By controlling the data, they control us.''

N.B.A. general managers and team owners are not allowed to see the league's referee statistics, several people familiar with the rating system said. Three months ago, while disputing an academic study that said white and black referees called fouls at varying rates on players of different races, Joel Litvin, the N.B.A.'s president for basketball operations, defended keeping the information private.

''Even our teams don't know the data we collect as to a particular referee's call tendencies on certain types of calls,'' Litvin said at the time. ''There are good reasons for this. It's proprietary. It's personnel data at the end of the day.''

Litvin declined comment while leaving yesterday's news conference.

The N.B.A.'s system of monitoring in-game officials contrasts with that of Major League Baseball.

According to one major league umpire, who spoke on condition of anonymity because he did not want to appear to be speaking about Donaghy or N.B.A. policy, umpires receive and review great amounts of information regarding their performance. Most of it, he said, involves percentages of correct ball and strike calls as determined by QuesTec, the computerized video system Major League Baseball has installed in more than 10 parks. QuesTec grades are used for grading but not evaluated for job retention.

The umpire said that he and his colleagues were routinely given computer disks from Major League Baseball that break down their percentages for any game that QuesTec cameras recorded. Umpires that fall below 90 percent accuracy do not meet the minimum standard. If an umpire falls below that level, the league office works with him to improve his calls. The umpire said that the disks also included video from several angles, as well as video replays for other decisions -- like calls at bases -- for the umpires to review.

''I think it's now a good thing,'' said the umpire, referring to how umpires originally objected to the system as imperfect and intrusive. ''The staff has accepted it. We're our own worst critics. We want to get the calls right.''

In the National Football League, calls are monitored every week. Mike Pereira, the league's vice president for officiating, spends game days in the league's officiating control center monitoring every game on high definition televisions. On Monday mornings, Pereira provides Commissioner Roger Goodell with a tape of critical or questionable plays. Later in the week, Pereira and other supervisors grade the officiating crews. Individual refs are then told of their mistakes, and those instances are discussed.

Photo: The N.B.A. coach Maurice Cheeks pleaded his case to the referee Tim Donaghy in 2004. Commissioner David Stern pledged to re-evaluate how referees are monitored. (Photograph by Donna McWilliam/Associated Press)