Bedrock

AWS Bedrock provides access to foundation models from Amazon, Anthropic, Meta, Mistral, and more through a fully managed service.

Bedrock is an optional provider. Install with: pip install ai-query[bedrock]

Setup

Option 1: Default Credentials (Recommended)

Use the standard AWS credential chain (env vars, ~/.aws/credentials, IAM roles). If no session is provided, ai-query creates a default boto3 client.

export AWS_REGION="us-east-1"
export AWS_ACCESS_KEY_ID="AKIA..."
export AWS_SECRET_ACCESS_KEY="..."

from ai_query.providers.bedrock import bedrock

model = bedrock("anthropic.claude-3-5-sonnet-20241022-v2:0")

Option 2: Custom Session (SSO, Profiles, AssumeRole)

For advanced authentication, create a boto3.Session and pass it to the provider. This gives you full control over credentials. Using a named profile (SSO):

import boto3
from ai_query.providers.bedrock import bedrock

# Create session from profile
session = boto3.Session(profile_name="my-sso-profile")

model = bedrock(
    "anthropic.claude-3-5-sonnet-20241022-v2:0",
    session=session
)

Using Assumed Role (via STS):

import boto3
from ai_query.providers.bedrock import bedrock

# 1. Assume role
sts = boto3.client("sts")
response = sts.assume_role(
    RoleArn="arn:aws:iam::123456789012:role/my-bedrock-role",
    RoleSessionName="ai-query-session"
)
creds = response["Credentials"]

# 2. Create session with temporary credentials
session = boto3.Session(
    aws_access_key_id=creds["AccessKeyId"],
    aws_secret_access_key=creds["SecretAccessKey"],
    aws_session_token=creds["SessionToken"],
    region_name="us-east-1"
)

# 3. Use session
model = bedrock(
    "anthropic.claude-3-5-sonnet-20241022-v2:0",
    session=session
)

from ai_query.providers.bedrock import bedrock

model = bedrock(
    "anthropic.claude-3-5-sonnet-20241022-v2:0",
    region="us-east-1",
    credentials={
        "access_key": "AKIA...",
        "secret_key": "...",
        "session_token": "..."  # optional
    }
)

Usage

from ai_query import generate_text
from ai_query.providers.bedrock import bedrock

result = await generate_text(
    model=bedrock("anthropic.claude-3-5-sonnet-20241022-v2:0"),
    prompt="Explain machine learning in simple terms."
)

Available Models

Model ID	Description
`anthropic.claude-3-5-sonnet-20241022-v2:0`	Claude 3.5 Sonnet v2
`anthropic.claude-3-haiku-20240307-v1:0`	Claude 3 Haiku
`meta.llama3-1-70b-instruct-v1:0`	Llama 3.1 70B Instruct
`meta.llama3-1-8b-instruct-v1:0`	Llama 3.1 8B Instruct
`mistral.mistral-large-2407-v1:0`	Mistral Large
`amazon.nova-pro-v1:0`	Amazon Nova Pro
`amazon.nova-lite-v1:0`	Amazon Nova Lite

See the AWS Bedrock Model IDs page for the full list of supported models.

Provider Options

Customize Bedrock-specific parameters:

result = await generate_text(
    model=bedrock("anthropic.claude-3-5-sonnet-20241022-v2:0"),
    prompt="Write a story.",
    provider_options={
        "bedrock": {
            "temperature": 0.7,
            "max_tokens": 2048,
            "top_p": 0.9,
            "stop_sequences": ["END"]
        }
    }
)

Available Options

Option	Type	Description
`temperature`	`float`	Randomness (0-1)
`max_tokens`	`int`	Maximum output tokens
`top_p`	`float`	Nucleus sampling
`stop_sequences`	`list[str]`	Stop generation at these sequences

Tool Calling

Bedrock supports tool calling through the Converse API:

from ai_query import generate_text
from ai_query.providers.bedrock import bedrock
from ai_query import tool, Field

@tool(description="Get current weather")
async def get_weather(
    location: str = Field(description="City name")
) -> str:
    return f"Weather in {location}: 72°F, sunny"

result = await generate_text(
    model=bedrock("anthropic.claude-3-5-sonnet-20241022-v2:0"),
    prompt="What's the weather in Seattle?",
    tools={"get_weather": get_weather}
)

Streaming

from ai_query import stream_text
from ai_query.providers.bedrock import bedrock

result = stream_text(
    model=bedrock("anthropic.claude-3-5-sonnet-20241022-v2:0"),
    prompt="Write a poem about coding."
)

async for chunk in result.text_stream:
    print(chunk, end="", flush=True)

Why Bedrock?

Managed service: No infrastructure to manage
Multiple providers: Access models from Anthropic, Meta, Mistral, Amazon
VPC support: Run in your private network
Model customization: Fine-tune models with your data
Pay-per-use: No upfront commitments

Overview

Native Providers

OpenAI-Compatible

Cloud Providers

Setup

Option 1: Default Credentials (Recommended)

Option 2: Custom Session (SSO, Profiles, AssumeRole)

Usage

Available Models

Provider Options

Available Options

Tool Calling

Streaming

Why Bedrock?

​Setup

​Option 1: Default Credentials (Recommended)

​Option 2: Custom Session (SSO, Profiles, AssumeRole)

​Usage

​Available Models

​Provider Options

​Available Options

​Tool Calling

​Streaming

​Why Bedrock?

Setup

Option 1: Default Credentials (Recommended)

Option 2: Custom Session (SSO, Profiles, AssumeRole)

Usage

Available Models

Provider Options

Available Options

Tool Calling

Streaming

Why Bedrock?