Non-LLM transformers beat tree boosting - TabPFN, TabFM
https://research.google/blog/introducing-tabfm-a-zero-shot-f...
*on small datasets
*on small datasets