Valeriy’s Substack

Valeriy’s Substack

Ordered boosting in one walkthrough: how CatBoost makes gradient boosting unbiased

Valeriy Manokhin's avatar
Valeriy Manokhin
May 16, 2026
∙ Paid

Previous post argued that standard gradient boosting has a self-referential bias problem: every example contributes to the model that then predicts it. Cross-validation under-detects the bias because it manifests inside a single training fold.

User's avatar

Continue reading this post for free, courtesy of Valeriy Manokhin.

Or purchase a paid subscription.
© 2026 Valery Manokhin · Privacy ∙ Terms ∙ Collection notice
Start your SubstackGet the app
Substack is the home for great culture