Computers put on editing cap - and cut out the thinking

March 30 2002

New technology that can sort articles and write news summaries is being tested with mixed results, Susan Reed reports.

Reporters whose work appears on news Web sites are finding that yet another editor is going through their articles after they are published. This one is unseen and unheard and cannot be reasoned with. It can do an impressive job, or a disturbing one, of changing their words. And it never issues a correction.

This editor works only in cyberspace, as an experiment by computer scientists in the natural-language programming group at Columbia University, who have created a software program for editing, summarising and writing called Newsblaster.

While software had long been able to sort information, Columbia's project was among the most significant devoted to summarising large streams of data, said Christopher Manning, who teaches computer science and linguistics at Stanford University.

Five years in the making, Newsblaster has been supported by grants from the National Science Foundation and the Defence Advanced Research Projects Agency, which is seeking ways to make it easier for intelligence analysts to browse for information.

But the software also has the potential to help consumers manage the sometimes overwhelming amount of information generated by news organisations and to update them on the latest developments.");document.write("

advertisement

");
}
}
// -->

The program at present trawls 17 Web sites, including those of The Washington Post, Reuters and the BBC, gathering articles and sorting them by topic. Once Newsblaster has organised the articles by subject, it begins to search for shared terms and common phrases that it can use to produce a five-sentence summary. (These can be read at www.cs.columbia.edu/nlp/newsblaster)

An aerospace reporter for the Associated Press, Marcia Dunn, who is based in Cape Canaveral, noted that in rewriting and tightening her copy, Newsblaster had introduced inaccuracies in a story on the Hubble space telescope.

But despite the hiccups, "I would give it a good grade", she said.

The eight-member group that created Newsblaster is led by Kathleen McKeown, the chairwoman of Columbia's computer science department.

She said that once Newsblaster had categorised articles by theme, it "analyses sentences to find the noun phrase, the verb phrase, the subject, the object, and it will do a comparison across them to find out how much of it is similar".

It then wrote its summary by combining phrases from those sentences.

Although Professor McKeown said Newsblaster was trained to recognise news articles, mainly by length, it could not determine that an opinion column was not news and does not necessarily note opposing views on an issue. She estimated the program was 88per cent accurate.

The Newsblaster program is still in the research phase, but Professor Manning predicts that ultimately the perfection of Newsblaster will hand human editing over to a machine.

Dozens of software and news companies, many of which are in the Fortune 500, have expressed an interest in licensing the program.