Prompt Engineering vs RAG vs Fine tuning.
Let us explore the difference between prompt engineering, RAG, and fine-tuning.
|
Criteria |
Prompt Engineering |
RAG |
Fine-Tuning |
|---|---|---|---|
|
Purpose |
Prompt engineering focuses on how to write an effective prompt that can maximize the generation of an optimized output for a given task. |
The purpose of RAG is to relevant information for a given prompt from an external database. |
Fine-tuning focuses on training and adapting a model for a specific task. |
|
Model |
Model weights are not updated. It focuses on building an effective prompt. |
Model weights are not updated. It focuses on building context for a given prompt. |
Model weights are updated |
|
Complexity |
No technical knowledge required |
Compared to fine-tuning it is less complex as it requires skills related to vector databases and retrieval mechanisms only |
Technical knowledge required |
|
Compute Cost |
Very less cost. Only costs related to API calls |
Cost-effective compared to fine-tuning. |
We may need specialized hardware to train the model depending on model size and dataset size |
|
Knowledge |
The model does not learn new data |
The prompt is equipped with new data in the form of context |
The model learns new data |
Fine Tuning Large Language Model (LLM)
Large Language Models (LLMs) have revolutionized the natural language processing by excelling in tasks such as text generation, translation, summarization and question answering. Despite their impressive capabilities, these models may not always be suitable for specific tasks or domains due to compatibility issues. To overcome this fine tuning is performed. Fine tuning allows the users to customize pre-trained language models for specialized tasks. This involves refining the model on a limited dataset of task-specific information, enhancing its performance in that particular task while retaining its overall language proficiency.
Table of Content
- What is Fine Tuning?
- Why Fine-tune?
- Types of Fine Tuning
- Prompt Engineering vs RAG vs Fine tuning.
- When to use fine-tuning?
- How is fine-tuning performed?
- Fine Tuning Large Language Model Implementation