Foundation Models

A Foundation model undergoes broad pretraining, often through self-supervised learning, and is then adapted to a downstream task through methods such as fine-tuning.

Japanese terms

1. Foundation model — 基盤きばんモデル: A model trained broadly at scale that can be adapted to many downstream tasks.
2. Pretraining — 事前学習じぜんがくしゅう: Initial training on broad data before a model is adapted for a more specific purpose.
3. Self-supervised learning — 自己教師じこきょうしあり学習がくしゅう: Learning in which supervisory signals are derived from the structure of the input data itself.
4. Downstream task — 下流かりゅうタスク: A specific application for which a pretrained model is adapted or evaluated.
5. Fine-tuning — ファインチューニング: Further training a pretrained model on task-specific data or objectives.

Related topics

Large language models are prominent text-oriented foundation models; multimodal AI extends the paradigm across data types.