Recent Comments

Archives

Categories

Meta

Welcome

Author: FJoch

This is a tiny toolkit for performing training of maximum entropy models. It is a generic toolkit, which is applicable to conditional maximum entropy models. A special feature of this program is that it is written only in 132 lines of code (can be printed on one sheet of paper …). It includes training of model parameters, evaluation, perplexity and error rate computation, count-based feature reduction, Gaussian priors, feature count normalization. In addition, it is easy to use and efficient enough to deal with millions of features.

YASMET is free software under the GNU Public License and is distributed without any warranty.

GIZA++ is an extension of the program GIZA (part of the SMT toolkit EGYPT) which was developed by the Statistical Machine Translation team during the summer workshop in 1999 at the Center for Language and Speech Processing at Johns-Hopkins University (CLSP/JHU). GIZA++ includes a lot of additional features. The extensions of GIZA++ were designed and written by Franz Josef Och.

Correct implementation of pegging as described in (Brown et al. 1993), a series of heuristics in order to make pegging sufficiently efficient;

…

In order to compile GIZA++ you may need:

a recent version of the GNU compiler (2.95 or higher)

a recent version of assembler and linker which do not have restrictions with respect to the length of symbol names

It is known to compile on Linux, Irix and SUNOS systems. A lot of older compiler version do not fully support all features of STL that are used by GIZA++. Therefore, frequently occur compiler, assembler or linker problems which are mostly due to the intensive use of STL within the program. If any compilation problem occurs, please first try to get the newest compiler version. Patches to the code are most welcome. Feel free to send me mail asking for help, but please do not necessarily expect me to have time to help.

Acknowledgements

This work was supported by the National Science Foundation under Grant No. No. IIS-9820687 through the 1999 Workshop on Language Engineering, Center for Language and Speech Processing, Johns Hopkins University.

mkcls is a tool to train word classes by using a maximum-likelihood-criterion. The resulting word classes are especially suited for language models or statistical translation models. The program mkcls was written by Franz Josef Och.