https://kobybibas.github.io/tags/in-context-learning/Recent content in In-Context Learning on Koby BibasHugo -- gohugo.ioenSun, 12 Jul 2026 00:00:00 +0000https://kobybibas.github.io/posts/20260712_tabular_foundation_models/summary/Sun, 12 Jul 2026 00:00:00 +0000https://kobybibas.github.io/posts/20260712_tabular_foundation_models/summary/TL;DR Decision tree models have dominated machine learning for tabular data, and previous attempts to apply transformers to tabular data ended with no success. Recently, a new paradigm has emerged: train a large transformer model on synthetic datasets, then apply in-context learning to a new dataset to predict the entire test set labels. Multiple recent papers (TabPFN, TabICL, TabFM) show this approach outperforms tuned decision-tree models. However, per-prediction inference is actually slower than trees.