Skip to main content
Groq provides ultra-fast inference for open-source models using their custom LPU (Language Processing Unit) hardware. Their API is OpenAI-compatible for seamless integration.

Setup

Set your API key as an environment variable:
Get your API key from Groq Console.

Usage

Available Models

See the Groq models page for a full list.

Provider Options

Customize parameters:

Streaming

Why Groq?

  • Ultra-fast inference: LPU hardware delivers industry-leading speed
  • Open models: Access to Llama, Mixtral, Gemma and more
  • Free tier: Generous free usage for experimentation
  • Low latency: Great for real-time applications
  • OpenAI compatible: Easy migration from OpenAI