LLM Fine-tuning Use Case

Content Summarization

Summarizing key information from unstructured text has applications across many domains like customer support, healthcare, recruiting, and more. However, manually creating clear and accurate summaries is time-consuming and requires expert knowledge — that’s where LLMs come in. Learn how to fine-tune a small open-source language model to automatically generate highly accurate summaries for any use case.

The Predibase Solution

Generate accurate summaries with LLMs.

Fine-tune a small language model (Llama-3.1-8b) on a dataset of dialogues.
Instantly deploy and query your high-quality dialogue summarizer on serverless endpoints.
Use automatically generated summaries to reduce the time needed to interpret large volumes of text, thereby speeding up decision-making.

Unstructured Text

Conversations/Dialogues

Call Transcripts

Long Documents

Contract Proposals

Chats/Emails

Automatic Summarization with LLMs

Example Use Cases

Summarizing patient records and clinical trial reports

Summarizing customer support tickets and chat logs

Summarizing lengthy legal documents and contracts

Fine-tune and Serve your Own Small Language Model for Text Summarization

Easily and efficiently fine-tune open-source Meta-Llama-3.1-8B-Instruct to generate high-quality summaries of lengthy conversations. Instantly serve and prompt your fine-tuned LLM for dialogue summarization cost-effectively with endpoints built on top of open-source LoRAX.

# Create the adapter repository
repo = pb.repos.create(
    name="samsum-llama-3-1-8b-instruct",     description='Fine-tuning "llama-3-1-8b-instruct" on "samsum" dataset with Predibase.',
    exists_ok=True,
)

# Kick off the fine-tuning job
adapter_real: FinetuningJob = pb.finetuning.jobs.create(
    config=FinetuningConfig(
        base_model="samsum-llama-3-1-8b-instruct",
        epochs=5,
        learning_rate=0.0002,
    ),
    dataset=fine_tuning_dataset,
    repo="samsum-llama-3-1-8b-instruct",
    description='Fine-tune "llama-3-1-8b-instruct" on "samsum_5000_rows" dataset.',
)  

# Dynamically load fine-tuned adapter for inference
client.generate(
    control_prompt,
    # adapter version 1 is used (corresponds to the number of fine-tuning run)
    adapter_id="samsum-llama-3-1-8b-instruc/1",
    max_new_tokens=256,
).generated_text

Example code in Predibase

Performance Evaluation: Predibase vs. GPT-4

Small language models fine-tuned on Predibase outperform GPT-4. The following diagram shows the performance of our fine-tuned SLM (Llama-3.1-8B-Instruct) compared to the base model and GPT-4 using the ROUGE-1 metric. The fine-tuned SLM exhibits a 22%improvement over GPT-4.

Resources to Get Started

Ready to efficiently fine-tune and serve your own LLM?

Try Predibase for Free