Table foundation models are pre-trained on tabular data to enhance knowledge and priors for downstream tasks on tables.
Challenges in dealing with data semantics in numerical entries are addressed by pre-trained neural networks that model column names and table entries together.
TARTE is introduced as a foundation model that uses strings to capture semantics, transforming tables into knowledge-enhanced vector representations pre-trained on large relational data.
TARTE's representations can be fine-tuned or combined with other learners to improve prediction accuracy and computation performance, making it an effective approach to knowledge pre-training for tabular learning.