Python SDK

Complete guide to the ModelRed Python SDK — installation, configuration, and usage patterns.

Introduction

The ModelRed Python SDK provides both synchronous and asynchronous clients for integrating LLM security assessments into your workflow. Built with automatic retries, comprehensive error handling, and type hints for the best developer experience.

Installation

Install via pip or uv:

pip install modelred
uv add modelred

The SDK requires Python 3.8 or higher and depends on httpx for HTTP communication.

Client Initialization

Synchronous Client

from modelred import ModelRed

client = ModelRed(
    api_key="mr_...",
    timeout=20.0,
    max_retries=3,
)

Asynchronous Client

from modelred import AsyncModelRed

async with AsyncModelRed(api_key="mr_...") as client:
    # Use client here
    pass

Configuration Options

PropTypeDefault
api_key?
string
-
timeout?
float
20.0
max_retries?
int
3
transport?
httpx.BaseTransport | None
-
extra_headers?
dict[str, str] | None
-

Context Manager Pattern

Both clients support context managers for automatic resource cleanup:

    with ModelRed(api_key="mr_...") as client:
        models = client.list_models()
        # client.close() called automatically
    async with AsyncModelRed(api_key="mr_...") as client:
        models = await client.list_models()
        # await client.aclose() called automatically

Manual Cleanup

If not using context managers, remember to close the client:

    client = ModelRed(api_key="mr_...")
    try:
        # Use client
        models = client.list_models()
    finally:
        client.close()
    client = AsyncModelRed(api_key="mr_...")
    try:
        # Use client
        models = await client.list_models()
    finally:
        await client.aclose()

Retries & Backoff

The SDK automatically retries requests for transient failures:

Retryable Errors

429 — Rate limited (exponential backoff with jitter)
502, 503, 504 — Server errors
Transport errors — Network issues, connection failures

Backoff Formula

delay = random.uniform(0, min(8.0, 0.5 * 2^attempt))
Base: 0.5s, Cap: 8s

Configure retry behavior:

# Default: 3 retries with 0.5s base, 8s cap
client = ModelRed(api_key="mr_...", max_retries=3)

# Disable retries
client = ModelRed(api_key="mr_...", max_retries=0)

# Aggressive retries
client = ModelRed(api_key="mr_...", max_retries=5)

Automatic Retries: The SDK handles rate limits and transient errors automatically. You typically won't see 429 or 5xx errors unless retries are exhausted.

Base URL

The SDK is configured to use https://www.app.modelred.ai as the base URL. This is fixed and cannot be changed in the current version.

User Agent

All requests include a user agent header identifying the SDK version:

modelred-python-sdk/0.1.39

Type Hints

The SDK is fully typed and includes type hints for all public methods. Use static type checkers like mypy or pyright for enhanced development experience:

from modelred import ModelRed
from typing import Dict, Any

client: ModelRed = ModelRed(api_key="mr_...")
assessment: Dict[str, Any] = client.get_assessment("assessment_id")

API Key Requirements

Your API key must:

  • Start with mr_
  • Be valid and active in your ModelRed organization
  • Have appropriate permissions for the operations you're performing

The SDK validates the key format on initialization:

# Raises ValueError
client = ModelRed(api_key="invalid_key")

# Raises ValueError
client = ModelRed(api_key="")

Thread Safety

The synchronous client uses httpx's Client, which maintains a connection pool but is not thread-safe by default. For concurrent requests:

    from modelred import ModelRed
    import threading
    
    client = ModelRed(api_key="mr_...")
    
    # NOT RECOMMENDED: Shared client across threads
    def worker():
        models = client.list_models()
    
    threads = [threading.Thread(target=worker) for _ in range(5)]
    for t in threads:
        t.start()
    for t in threads:
        t.join()
client per thread def worker(): client = ModelRed(api_key="mr_...") models =
client.list_models() client.close() threads = [threading.Thread(target=worker)
for _ in range(5)] for t in threads: t.start() for t in threads: t.join() ```
</Tab>

<Tab value="Async">
```python
  import asyncio
  from modelred import AsyncModelRed
  
  async def worker(client: AsyncModelRed):
      models = await client.list_models()
  
  async def main():
      async with AsyncModelRed(api_key="mr_...") as client:
          await asyncio.gather(*[worker(client) for _ in range(5)])
  
  asyncio.run(main())

Complete Examples

Synchronous Workflow

sync_example.py
from modelred import ModelRed
import os

# Initialize client
client = ModelRed(
    api_key=os.environ["MODELRED_API_KEY"],
    timeout=30.0,
    max_retries=3,
)

try:
    # List models
    models = client.list_models(provider="openai", status="active")
    print(f"Found {len(models['data'])} models")

    # Get probe packs
    owned = client.list_owned_probes(category="injection", page_size=5)

    # Create assessment
    if models["data"] and owned["data"]:
        assessment = client.create_assessment_by_id(
            model_id=models["data"][0]["id"],
            probe_pack_ids=[owned["data"][0]["id"]],
            detector_provider="openai",
            detector_api_key=os.environ["OPENAI_API_KEY"],
            detector_model="gpt-4o-mini",
        )
        print(f"Assessment created: {assessment['id']}")

finally:
    client.close()

Asynchronous Workflow

async_example.py
import asyncio
from modelred import AsyncModelRed
import os

async def main():
    async with AsyncModelRed(
        api_key=os.environ["MODELRED_API_KEY"],
        timeout=30.0,
        max_retries=3,
    ) as client:
        # List models
        models = await client.list_models(provider="anthropic", status="active")
        print(f"Found {len(models['data'])} models")

        # Get probe packs
        imported = await client.list_imported_probes(category="jailbreak", page_size=5)

        # Create assessment
        if models["data"] and imported["data"]:
            assessment = await client.create_assessment_by_id(
                model_id=models["data"][0]["id"],
                probe_pack_ids=[imported["data"][0]["id"]],
                detector_provider="anthropic",
                detector_api_key=os.environ["ANTHROPIC_API_KEY"],
                detector_model="claude-3-5-sonnet-20241022",
            )
            print(f"Assessment created: {assessment['id']}")

asyncio.run(main())

Environment Configuration

Use environment variables for sensitive configuration:

config.py
import os
from modelred import ModelRed
from dotenv import load_dotenv

# Load from .env file
load_dotenv()

# Create client from environment
client = ModelRed(
    api_key=os.environ["MODELRED_API_KEY"],
    timeout=float(os.environ.get("MODELRED_TIMEOUT", "20.0")),
    max_retries=int(os.environ.get("MODELRED_MAX_RETRIES", "3")),
)

Example .env file:

MODELRED_API_KEY=mr_your_key_here
MODELRED_TIMEOUT=30.0
MODELRED_MAX_RETRIES=5
OPENAI_API_KEY=sk_your_openai_key
ANTHROPIC_API_KEY=sk-ant_your_anthropic_key

Security: Never commit .env files to version control. Add .env to your .gitignore file.

Custom Transport

Configure custom HTTP transport for proxies or testing:

import httpx
from modelred import ModelRed

# Corporate proxy
transport = httpx.HTTPTransport(
    proxy="http://proxy.company.com:8080"
)

client = ModelRed(
    api_key="mr_...",
    transport=transport,
)

Custom Headers

Add custom headers to all requests:

from modelred import ModelRed

client = ModelRed(
    api_key="mr_...",
    extra_headers={
        "X-Custom-Header": "value",
        "X-Request-ID": "unique-id",
    },
)

Error Handling

The SDK provides comprehensive error types:

from modelred import (
    ModelRedError,      # Base exception
    APIError,           # HTTP/API errors
    Unauthorized,       # 401
    Forbidden,          # 403
    NotFound,           # 404
    ValidationFailed,   # 400, 422
    RateLimited,        # 429
    ServerError,        # 5xx
)

try:
    assessment = client.get_assessment("assessment_id")
except NotFound as e:
    print(f"Assessment not found: {e.message}")
except APIError as e:
    print(f"API error {e.status}: {e.message}")

See the Error Handling guide for comprehensive error management.

Best Practices

Security

Never hardcode API keys. Use environment variables or secrets managers for production deployments.

Performance

Reuse client instances to benefit from connection pooling. Use async client for concurrent operations.

Resource Management

Always use context managers or ensure manual cleanup with close() or aclose().

# ✅ Good: Reuse client
client = ModelRed(api_key=os.environ["MODELRED_API_KEY"])

def get_models():
return client.list_models()

def create_assessment():
return client.create_assessment_by_id(...)

# ❌ Bad: Creates new client each time

def get_models():
client = ModelRed(api_key=os.environ["MODELRED_API_KEY"])
return client.list_models()

Next Steps

Now that you understand client configuration, explore specific operations:

Overview

ModelRed — production-grade LLM security assessments, probe packs, and operational visibility.

Assessments

Create, monitor, and manage LLM security assessments with the ModelRed SDK.