Main menu

Monthly Archives: March 2013

In information retrieval and text classification, tf-idf plays a big role. Read the Wikipedia article to learn what it is about, here I want to deal with the problem of typesetting the formula in LaTeX.

The formula is log-weighted term frequency tf times inverse document frequency idf, if we naivly write this down, we arrive at this:

tf-idf_{t,d} = (1 +\log tf_{t,d}) \cdot \log \frac{N}{df_t}

When you look at the LaTeX output, you will see that several things go wrong. In math mode, LaTeX interprets two letters next to each other as a product of two variables. So the name tf becomes the mathematical expression “t times f” and is typeset accordingly. Also, in case of tf-idf, the name contains a hyphen. In math mode a hyphen between two expression is interpreted as a minus sign. So this is definitely not what we want.

How do we solve the problem? What we want is that this part is interpreted as normal text. One possibility to add text to equations is the command \mbox{} (another is the command \text{} which requires the amsmath package). So this is it:

normalizeParentheses: Whether to map round parentheses to -LRB-, -RRB-, as in the Penn Treebank

normalizeOtherBrackets: Whether to map other common bracket characters to -LCB-, -LRB-, -RCB-, -RRB-, roughly as in the Penn Treebank

untokenizable: What to do with untokenizable characters (ones not known to the tokenizer). Six options combining whether to log a warning for none, the first, or all, and whether to delete them or to include them as single character tokens in the output: noneDelete, firstDelete, allDelete, noneKeep, firstKeep, allKeep. The default is “firstDelete”.

escapeForwardSlashAsterisk: Whether to put a backslash escape in front of / and * as the old PTB3 WSJ does for some reason (something to do with Lisp readers??).

Sometimes you have a LaTeX beamer presentation and want to have some "backup" slides that you may show if the audience is really interested in this detail, but otherwise not. There is a simple solution for that, the package appendixnumberbeamer.

You need to load the package in the preamble:

\usepackage{appendixnumberbeamer}

Then you just need to use "appendix" before the slides you want to have as backup:

The slides in the appendix will not count towards the total slide number that is displayed for the normal slides. Backup slides will have their own slide numbers and total slide numbers counted anew from the start of the appendix. Very handy!

You can organize your backup slides in sections, these section will not appear in the table of content. If you use a beamer template with navigation (miniframes like in Szeged, or split like in Malmoe), the backup slides will not appear in the navigation. A cool thing is that on the backup slides, the navigation will show the structure of the backup slides, so you can easily change to the slide you want. A disadvantage is of course that everybody will see that you have more backup slides than actual slides 😉