Whisper: OpenAI's Multilingual Speech Recognition Model

Overview

Whisper is OpenAI's open-source, multilingual automatic speech recognition (ASR) model capable of transcription and translation without fine-tuning.

Key Features

• Multilingual support
• Multitasking (transcription, translation, language identification)
• Open-source availability
• No fine-tuning required

How Whisper Works

Whisper uses large-scale weak supervision for training, enabling it to perform well across various languages and tasks.

Model Variants

Size	Parameters	Description
Tiny	39 M	Fastest, lowest accuracy
Base	74 M	Balanced speed and accuracy
Small	244 M	Improved accuracy
Medium	769 M	High accuracy
Large	1.5 B	Highest accuracy, slowest

Using Whisper with LangChain

LangChain provides an easy way to integrate Whisper for audio processing:

from langchain.document_loaders.generic import GenericLoader
from langchain.document_loaders.parsers import OpenAIWhisperParser
from langchain.document_loaders.blob_loaders.youtube_audio import YoutubeAudioLoader

url = "https://www.youtube.com/watch?v=-6Hu9_NBlOs"
save_dir = "/path/to/save/audio/"

loader = GenericLoader(
    YoutubeAudioLoader([url], save_dir),
    OpenAIWhisperParser()
)

docs = loader.load()

# Print transcribed content
for doc in docs:
    print(doc.page_content[:500])
    print("---")