#foundationalmodels [[Future of Foundational Models]] [[ChatGPT]] # What are foundational models? A foundation model is a "**paradigm for building AI systems" ** in which a model trained on a large amount of unlabeled data can be adapted to many applications. Foundational models act as a platform for creating [[specialised models]]. They are designed to be adapted to various downstream cognitive tasks by pre-training on broad data at scale. They can also be effective for tasks for which it has not previously been trained. For eg. BERT, CLIP, DALL-E, and GPT-3 ![[Pasted image 20230104083330.png]] # How can a foundation model be adapted? [Transfer learning](https://research.aimultiple.com/transfer-learning/) is the ML technique that enables the emergence of foundation models. Based on the accumulated knowledge gained at previous tasks, a model can learn new tasks through this technique. Foundation models need to be adapted because they serve as a base for new models, and there are numerous approaches to do this, such as: ### Fine-tuning This is the process of adopting a given model to meet the needs of a different task. Thus, instead of generating a new model for this purpose, a modification will suffice. ### In-context learning Using this approach, models can learn how to perform a task with minimum training and without fine-tuning, unlike conventional approaches. # What are the applications of foundation models? The foundation models can be applied to a wide range of industries, including healthcare, education, translation, social media, law, and more.  The following are the use cases that exist in all those industries:   - E-mail generation - Content creation - Text summarization - Translation - Answering questions  - Customer support - Website creation - Object tracking - Image generation & classification ![[Pasted image 20230104083412.png]] # What are the challenges of foundation models? While foundation models are referred to as “the new paradigm of AI” or “the future of AI,” there are serious obstacles in the widespread implementation of these models. - Unreliableness - Biases - Incomprehension ---- Sources: 1. https://arxiv.org/pdf/2108.07258.pdf 2. https://www.madrona.com/foundation-models-create-opportunity-tooling-layer/ 3. https://research.aimultiple.com/foundation-models/ 4. https://research.aimultiple.com/transfer-learning/ Link: - [[Truebit]] -