The open season for rating agency bashing is not over. Concerns about conflicts of interest and the value of their ratings on structured credit, which were sparked by the crisis last summer, were compounded in May by news that the computer systems they relied on to produce the ratings may have also been at fault. The industry has begun an investigation to determine the extent of the problem.

Moody’s Investors Service, led by chief executive Ray McDaniel, has started disciplinary proceedings against staff that gave triple-A ratings to complex debt vehicles that should have been rated lower, and has questioned the role computer software and systems played in the rating process.

A coding error in a computer model has been blamed for causing constant proportion debt obligations, complex credit derivatives products, to be rated higher than they should have been under the model. Moody’s rated 44 CPDO tranches, representing $4bn (€2.5bn) in rated securities.

Questions are growing over all the agencies’ use of computer models and whether there was an over-reliance on them.

The three main agencies, Standard & Poor’s, Fitch Ratings and Moody’s all use computer models to assess the risk of the securities in structured finance products they rate, but their criteria differ. They use different mathematical formulas, different levels of stress testing, and different assumptions in terms of historic default data.

Since the computer problem in rating the CPDOs appeared, Moody’s has centralized the responsibility for verifying key rating models in the credit policy department and is conducting a review of all primary models.

A spokesman for Moody’s in New York said: “We have not identified errors that would impact ratings.”

Models were continually updated and revised to “ensure accuracy and to respond to changes in the market,” the spokesman said.

Asked whether there was an over-reliance on computer models, the spokesman said: “It is important to recognize that computer models are an input to the rating process. Credit ratings are determined by a rating committee, not by computer models.”

Moody’s is also trying to include more independent analysts on ratings committees.

None of the three agencies would disclose which type of historical data, stress tests or mathematical formulas they use.

Arturo Cifuentes, managing director in structured finance at New York-based fixed income broker-dealer RW Pressprich, believes ratings should be based on clearly articulated methodologies and not black-box models. He said Moody’s computer models were based on an expected loss concept, or the probability of loss on which a basic premium rate is calculated.

Market sources said S&P and Fitch used a different approach for their computer models, based on the probability of default, or the likelihood that debt will not be repaid.

Ian Bell, head of European structured finance, and Joanne Rose, executive managing director of global structured finance at S&P, wrote in a report: “Our rating speaks to the likelihood of default, but not the amount that may be recovered in a post-default scenario.” As such, structured finance ratings are an opinion on the default risk associated with either an issuer or an issue, based on all the information available to the agency.

A spokesman for S&P declined to comment on how its models worked and whether it had implemented changes since last year.

The spokesman said: “Standard & Poor’s is committed to openness, transparency and helping restore confidence in the capital markets. As part of that commitment, we recently formed a separate model-validation group and hired a director of model quality to further enhance the regular testing of the models we use in our process. S&P will disclose any and all errors that result in a rating change so that the market has access to timely information.”

Vickie Tillman, executive director at S&P, last month wrote to the Securities and Exchange Commission acknowledging an error in a trial model of one of its computer models rating CPDOs, an S&P spokesman said. The SEC declined to comment.

A spokesman for Fitch, which also uses the probability of default approach, declined to elaborate on its models, saying that the agency was updating its policies, procedures and code, consistent with the proposals for rating agency reform made by the SEC last month.

Adding to the computer models’ complexity in all three agencies is the use of black boxes, computer program that receive a specific input and produce a given output. As their structure can be changed at any time and with the lack of knowledge about what parameters drive the output, it can be difficult for investors to interpret the meaning of the ratings, Cifuentes said. He said rating agencies should be transparent about the methodology and should not be in the business of developing software.

Cifuentes said: “The agencies should come up with mathematical formulas, equations and numerical assumptions, but the implementation of those, the software, should be done independently. If the only thing you do is show people a computer program and, not most of the methodology behind it, it is difficult to understand the rating.”

The use of historical data, a key parameter in the models, has also been problematic.

Given the nascent stage of the structured finance industry and therefore its constant innovation, there is a lack of data, and access to it is limited. Analysts often rely on issuers and bankers to collect data, because there often is no other source, generating another potential conflict of interest.