Wednesday, March 18, 2009

Adaboost : improve your weak performance

Adaboost is one of my favorite Machine Learning algorithm. The idea is quite intriguing: You start from a set of weak classifiers and learn how to linearly combine them so that the error is reduced. The result is a strong classifier built by boosting the weak classifiers.

Mathematically, given a set of labels y_i = { +1, -1 } and a training dataset x_i, Adaboost minimizes the exponential loss function sum_i (exp - (y_i * f(x_i)) }. The function f = sum_t (alpha_t h_t) is a linear combination of the h_t classifier with weight alpha_t. For those loving Optimization Theory , Adaboost is a classical application of Gradient Descend.The algorithm is quite simple and has been included in the top 10 data mining algorithms in 2007 and the Gödel prize in 2003. After Adaboost, Boosting become quite popular in the data mining community with application in Ranking and Clustering.

Hi, Nice effort!However, I am not able to compile it.I use a Mac I changed the -I to /opt/local/include and the -L /opt/local/lib However, I get the following error: g++ -L /opt/local/lib adaboost.o test.o -o ada_boostingld: in /opt/local/lib, can't map file, errno=22collect2: ld returned 1 exit status

Hey Nice post. I am looking for a document dealing with the practical aspects of Adaboost, like scaling the data, how to tweak the parameters like the number of weak classifiers, amount of training data and so on. Let me know if you are aware of such a document.ThanksJai Pillaihttp://www.umiacs.umd.edu/~jsp

hello sir,Im a student from India, currently working on a project "Automatic localization of Backward collision of vehicles using a single camera." I need your help in some of the terms, i have also read your paper on "Vehicle detection combining gradient analysis andAdaBoost classification". Its a wonderful work sir, i need your help in detecting objects(such as vehicles) at the rear end of a car. Im using haar tranforms and adaboost algorithm for the object detection. is this approach correct or do i need to make some changes?? kindly reply.