1、Bagging trains n base learner in parallel
2、Make decisions by averaging learers' outputs(regeression) or majority voting(classfication)
3、Each learner is trained bootstrap sampling
·Assume m training examples,than randomly sampling m examples with replacement
·Around 1-1/e≈63% examples will be sampled,the rest(out-of-bag) can be used for validation
class Bagging: def __init__(self, base_learner, n_learners): self.learners = [clone(base_learner) for _ in range(n_learners)] def fit(self, X,Y): for learner in self.learners: examples = np.random.choice(np.arange(len(X)), int(len(X)), replacement=True) learners.fit(X.iloc[examples,:],Y.iloc[examples]) def predict(self, X): preds = [learner.predict(X) for learner in self.learners] return np.array(preds).mean(axis=0)