MIToobox for C and MATLAB

This toolbox is aimed at people working on discrete datasets for classification. All functions expect discrete inputs.
It provides implementations of Shannon's Information Theory functions and implementations of Renyi's Entropy and Alpha Divergence. Version 2.0 includes weighted information theory functions based upon the work of S. Guiasu from "Information Theory with Applications" (1977). The toolbox was developed to
support our research into feature selection algorithms and includes some sample feature selection algorithms from the literature to illustrate its use.
Updated versions of the demonstration algorithms are provided (with many others) in the FEAST toolbox
we developed to support our research. A Java version of MIToolbox was ported from the C code, and is available here.

Note: all functions are calculated in log base 2, so return units of "bits".

Contains functions for:

Calculating Entropy, H(X)

Calculating Conditional Entropy, H(X|Y)

Calculating Mutual Information, I(X;Y)

Calculating Conditional Mutual Information, I(X;Y|Z)

Generating a joint random variable

Calculating Renyi's Alpha Entropy, H_{\alpha}(X)

Calculating Renyi's Alpha Mutual Information, I_{\alpha}(X;Y)

Calculating the Weighted Entropy, H_w(X)

Calculating the Weighted Conditonal Entropy, H_w(X|Y)

Calculating the Weighted Mutual Information, I_w(X;Y)

MATLAB Examples:

$ y = [1 1 1 0 0]';
$ x = [1 0 1 1 0]';

$ mi(x,y) %% mutual information I(X;Y)
ans =
0.0200

$ h(x) %% entropy H(X)
ans =
0.9710

$ condh(x,y) %% conditional entropy H(X|Y)
ans =
0.9510

$ h( [x,y] ) %% joint entropy H(X,Y)
ans =
1.9219

$ joint([x,y]) %% joint random variable XY
ans = [1,2,1,3,4]';

Also provided are example implementations of 3 feature selection algorithms (CMIM,
DISR, mRMR-D)
which use the functions provided by MIToolbox.
These example algorithms are provided in two forms, one coded in MATLAB and one coded in C using the MATLAB mex interface. The library is written in ANSI C for
compatibility with the MATLAB mex compiler.

All MIToolbox code is licensed under the LGPL v3, except the feature selection algorithms which are provided as is, with no warranty, for demonstration purposes.