AdaBoost vs. BayesianBoosting

cherokee

Hi!

I'm goint o try some boosting for my bachelor's thesis. I haven't yet decided whether using AdaBoost or BayesianBoosting. Actually I don't understand all the differences. What I do understand is that BayesianBoosing can use different fractions of the example set for model fitting and performance estimation. I understand that it is able to reweight examples to ensure equally distributed labels. But what exactly means [tt]allow_marginal_skews[/tt]?

Martin Scholz (the author of the operator) cites in the help text Scholz/2005b. Can anyone give me publication details on (t)his work. I think I would understand the differencen in detail if i read it.

Best regards,
chero

wessel

Hmm, let me first see if I understand the terminology correctly.

Bayesian Boosting is like creating all possible models and weighting them according their accuracy?
(Maybe you also need to weigh them according the model prior).
Most of the time this is not feasible in practice.

Adaboost, adaptive boosting, setting the iteration parameter to n, creates n models.
Models are weighted by their accuracy.
New training data is created by reweighing examples.
If examples are correctly classified in the previous iteration their weight goes down,
if examples are incorrectly classified their weight goes up.
Freund and Shaffire prove that the error on the training set goes down exponentially fast, using adaboost.

This paper is really good?
vorlon.case.edu/~sray/eecs600_fall08/ensembles_survey.ps
Ensemble Methods in Machine Learning Dietterich filetype:pdf

IngoRM

Hi,

you can find Martin's publication here:

http://www-ai.cs.uni-dortmund.de/auto?self=$Publication_e9zx9gcx

Here are his other publications on this and related topics:

http://www-ai.cs.uni-dortmund.de/PERSONAL/scholz.html

Cheers,
Ingo

wessel

This paper is about subgroup detection.

Not about ensembles or boosting.

IngoRM

Yes, but the basic algorithm - and hence also the operator - can be used for both problem types. It's the paper cited by Martin in his implementation comment so I assume it is the one he thinks it is most appropriate even if the title suggests something else. If you find another one of his papers more appropriate: just go ahead and cite it here. They all can be found on the web site stated above.

Cheers,
Ingo

Stefan_E

Ingo,

Thanks a lot for the links - was searching the same just over the weekend :-)

would propose to add these references to the wiki. There is a general lack of documentation where the various algorithms come from... so the wiki would be the natural place growing such a knowledge library.

Problem is that it appears that the wiki wants to be a duplication of the built-in help. If this is so, you'll end up in a maintenance problem as I expect there is only a one-way conversion path?

Stefan

IngoRM

Yes, you are right. There was actually a discussion about connecting the operator help and the Wiki some time ago:

http://rapid-i.com/rapidforum/index.php/topic,2013.msg8148.html#msg8148

But I must admit that I am not sure about the current state of this.

Cheers,
Ingo