Sessione Your Spreadsheet Is Not a Sentence: Foundation Models for Tables - M16 - Non solo LLM - Cloud Champions

Eventi>

Your Spreadsheet Is Not a Sentence: Foundation Models for Tables

(TeamSystem)
Lingua: Italiano
Orario: 20:00  -  20:45

Paste a CSV into a chatbot and ask it to make predictions. You'll likely get a confident answer, but one that often performs worse than an XGBoost model trained in five minutes. The reason is simple: language models treat numbers as text fragments and rows as sequences, while tabular data consists of numerical values and has no inherent order.

This talk explores how models designed specifically for tabular and time-series data overcome these limitations, comparing their approaches along three dimensions:

  • Architecture: TabPFN uses attention across rows and columns, while Chronos-2 introduces group attention to capture relationships between time series and their covariates.
  • Training: Instead of learning from web text, these models train on millions of synthetic datasets, an approach also adopted by Mistral's Timestral.
  • Inference: The training data becomes the prompt, allowing the model to generate predictions in a single forward pass, without gradient updates.


The results are compelling: with default settings, TabPFN-3.5 leads the TabArena leaderboard.

Yet important challenges remain, from scaling beyond a million rows to navigating benchmarks influenced by vendor claims and restrictive licenses, such as TimesFM 3's non-commercial terms.

By the end of the talk, attendees will understand how these models learn, how they make predictions, and where their strengths and limitations lie.