Error Handling | ModelRed

Comprehensive error types, HTTP status mapping, and recovery strategies for the ModelRed SDK.

Introduction

The ModelRed SDK provides a complete error hierarchy for handling API failures, validation issues, and operational problems. Understanding these errors helps you build robust, production-ready integrations.

Error Hierarchy

All SDK errors inherit from ModelRedError:

from modelred import (
    ModelRedError,      # Base exception
    APIError,           # HTTP/API errors
    Unauthorized,       # 401
    Forbidden,          # 403
    NotAllowedForApiKey,  # 403 (API key restrictions)
    NotFound,           # 404
    Conflict,           # 409
    ValidationFailed,   # 400, 422
    RateLimited,        # 429
    LimitExceeded,      # 403 (plan limits)
    ServerError,        # 5xx
)

Error Attributes

All API errors include structured information:

Prop	Type	Default
`status?`	`int`	-
`message?`	`string`	-
`code?`	`string \| None`	-
`details?`	`Any`	-

Access these attributes for detailed error handling:

from modelred import NotFound

try:
    assessment = client.get_assessment("nonexistent")
except NotFound as e:
    print(f"Status: {e.status}")           # 404
    print(f"Message: {e.message}")         # "Assessment not found"
    print(f"Code: {e.code}")               # "ASSESSMENT_NOT_FOUND"
    print(f"Details: {e.details}")         # Additional context

HTTP Status Mapping

400/422 — ValidationFailed

Invalid request parameters or malformed data:

from modelred import ValidationFailed

try:
    assessment = client.create_assessment_by_id(
        model_id="",  # Empty model ID
        probe_pack_ids=[],
        detector_provider="openai",
        detector_api_key="sk-...",
        detector_model="gpt-4o-mini",
    )
except ValidationFailed as e:
    print(f"Validation error: {e.message}")
    if e.details:
        print(f"Field errors: {e.details}")

Common causes: - Missing required fields - Invalid enum values - Empty arrays where items required - Malformed IDs or names

401 — Unauthorized

Invalid or missing API key:

from modelred import Unauthorized

try:
    client = ModelRed(api_key="invalid_key")
    models = client.list_models()
except Unauthorized as e:
    print(f"Authentication failed: {e.message}")
    # Check API key validity

Common causes: - Invalid API key format - Expired API key - Revoked API key - Missing API key

403 — Forbidden

Access denied. This can mean different things:

Insufficient permissions:

    from modelred import Forbidden
    
    try:
        # Attempting operation without permission
        result = client.some_admin_operation()
    except Forbidden as e:
        print(f"Access denied: {e.message}")

Plan or quota limits reached: python from modelred import LimitExceeded try: assessment = client.create_assessment_by_id( model_id="model_123", probe_pack_ids=["pack_1"], detector_provider="openai", detector_api_key="sk-...", detector_model="gpt-4o-mini", ) except LimitExceeded as e: print(f"Quota exceeded: {e.message}") # Upgrade plan or wait for quota reset

Operation requires web UI access:

    from modelred import NotAllowedForApiKey
    
    try:
        result = client.cancel_assessment("assessment_123")
    except NotAllowedForApiKey as e:
        print(f"Use web UI: {e.message}")
        # Direct user to web application

Common causes:

Plan limits (assessments, models, probe packs)
Rate limits (per-minute/per-hour caps)
Feature restrictions
API key vs. web UI operations

404 — NotFound

Resource does not exist:

from modelred import NotFound

try:
    assessment = client.get_assessment("nonexistent_id")
except NotFound as e:
    print(f"Not found: {e.message}")
    # Verify ID or check if resource was deleted

Common causes:

Invalid resource ID
Deleted resource
Typo in ID
Resource in different organization
Unimported public probe pack

409 — Conflict

Resource state conflict:

from modelred import Conflict

try:
    # Attempting conflicting operation
    result = client.some_operation()
except Conflict as e:
    print(f"Conflict: {e.message}")
    # Resolve state issue before retrying

Common causes:

Duplicate resource creation
Invalid state transition
Concurrent modification

429 — RateLimited

Too many requests:

from modelred import RateLimited
import time

try:
    models = client.list_models()
except RateLimited as e:
    print(f"Rate limited: {e.message}")
    time.sleep(5)  # Wait before retry
    # SDK already retries automatically

Note: The SDK automatically retries 429 errors with exponential backoff. You typically won't see these unless retries are exhausted.

5xx — ServerError

Server-side errors:

from modelred import ServerError

try:
    assessment = client.create_assessment_by_id(
        model_id="model_123",
        probe_pack_ids=["pack_1"],
        detector_provider="openai",
        detector_api_key="sk-...",
        detector_model="gpt-4o-mini",
    )
except ServerError as e:
    print(f"Server error: {e.message}")
    # Retry after delay or contact support

Note: 502, 503, and 504 errors are automatically retried by the SDK.

Comprehensive Error Handling

Catch All API Errors

from modelred import APIError

try:
    assessment = client.get_assessment("some_id")
except APIError as e:
    print(f"API error {e.status}: {e.message}")
    if e.code:
        print(f"Error code: {e.code}")

Catch All SDK Errors

from modelred import ModelRedError

try:
    client = ModelRed(api_key="mr_...")
    assessment = client.get_assessment("some_id")
except ModelRedError as e:
    print(f"ModelRed error: {e}")

Specific Error Handling

Handle different errors with specific recovery strategies:

from modelred import (
    ValidationFailed,
    Unauthorized,
    NotFound,
    LimitExceeded,
    RateLimited,
    ServerError,
)
import time

def create_assessment_with_retry(client, **kwargs):
    max_attempts = 3

    for attempt in range(max_attempts):
        try:
            return client.create_assessment(**kwargs)

        except ValidationFailed as e:
            # Don't retry validation errors
            print(f"Invalid parameters: {e.message}")
            raise

        except Unauthorized as e:
            # Don't retry auth errors
            print(f"Check API key: {e.message}")
            raise

        except NotFound as e:
            # Don't retry not found
            print(f"Resource not found: {e.message}")
            raise

        except LimitExceeded as e:
            print(f"Quota exceeded: {e.message}")
            # Could wait or upgrade plan
            raise

        except RateLimited as e:
            if attempt < max_attempts - 1:
                wait = 2 ** attempt
                print(f"Rate limited, waiting {wait}s...")
                time.sleep(wait)
                continue
            raise

        except ServerError as e:
            if attempt < max_attempts - 1:
                wait = 2 ** attempt
                print(f"Server error, retrying in {wait}s...")
                time.sleep(wait)
                continue
            raise

    raise Exception("Max retries exceeded")

Validation Error Details

ValidationFailed errors often include detailed field-level errors:

from modelred import ValidationFailed

try:
    assessment = client.create_assessment_by_id(
        model_id="",
        probe_pack_ids=[],
        detector_provider="invalid",
        detector_api_key="",
        detector_model="",
    )
except ValidationFailed as e:
    print(f"Validation failed: {e.message}")

    if e.details and isinstance(e.details, dict):
        if "fields" in e.details:
            for field, error in e.details["fields"].items():
                print(f"  {field}: {error}")

Automatic Retries

The SDK automatically retries certain errors:

Retryable Errors

429 — Rate Limited
502 — Bad Gateway
503 — Service Unavailable
504 — Gateway Timeout
Transport errors — Connection failures

Retry Behavior

Exponential backoff with jitter
Configurable max retries (default: 3)
Base delay: 0.5s, cap: 8s

# Configure retry behavior
client = ModelRed(
    api_key="mr_...",
    max_retries=5,  # More aggressive
)

# Disable retries

client = ModelRed(
api*key="mr*...",
max_retries=0, # No retries
)

Error Recovery Patterns

Retry with Backoff

import time
from modelred import RateLimited, ServerError

def with_backoff(func, max_attempts=3):
    for attempt in range(max_attempts):
        try:
            return func()
        except (RateLimited, ServerError) as e:
            if attempt == max_attempts - 1:
                raise
            wait = 2 ** attempt
            print(f"Retrying in {wait}s...")
            time.sleep(wait)

# Usage
result = with_backoff(
    lambda: client.get_assessment("assessment_123")
)

Fallback to Alternative

from modelred import NotFound

def get_assessment_or_create(client, assessment_id, **create_kwargs):
    try:
        return client.get_assessment(assessment_id)
    except NotFound:
        print("Assessment not found, creating new one...")
        return client.create_assessment(**create_kwargs)

Graceful Degradation

from modelred import APIError

def list_models_safe(client):
    try:
        return client.list_models(page_size=100)
    except APIError as e:
        print(f"Failed to list models: {e.message}")
        return {"data": [], "total": 0}  # Empty response

Async Error Handling

Error handling works identically for async operations:

import asyncio
from modelred import AsyncModelRed, NotFound, ValidationFailed

async def main():
    async with AsyncModelRed(api_key="mr_...") as client:
        try:
            assessment = await client.get_assessment("some_id")
        except NotFound as e:
            print(f"Not found: {e.message}")
        except ValidationFailed as e:
            print(f"Validation error: {e.message}")

asyncio.run(main())

Logging Errors

Integrate with Python's logging:

import logging
from modelred import APIError, ModelRed

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

client = ModelRed(api_key="mr_...")

try:
    assessment = client.get_assessment("some_id")
except APIError as e:
    logger.error(
        "API error occurred",
        extra={
            "status": e.status,
            "message": e.message,
            "code": e.code,
            "details": e.details,
        },
        exc_info=True,
    )

Testing Error Scenarios

Mock errors for testing:

import pytest
from modelred import NotFound, ValidationFailed
from unittest.mock import Mock

def test_handle_not_found():
    client = Mock()
    client.get_assessment.side_effect = NotFound(
        404, "Assessment not found", "NOT_FOUND", None
    )

    with pytest.raises(NotFound) as exc_info:
        client.get_assessment("missing_id")

    assert exc_info.value.status == 404
    assert "not found" in exc_info.value.message.lower()

Best Practices

Handle Expected Errors

Always catch specific errors you can recover from. Don't swallow exceptions without handling them properly.

Don't Catch Everything

Only catch errors you can actually handle. Let unexpected errors bubble up with full context.

Log Before Re-raising

Log error details before re-raising to maintain full error context for debugging.

Always Handle Expected Errors

# Good: Handle expected errors
try:
    assessment = client.get_assessment(assessment_id)
except NotFound:
    print("Assessment doesn't exist")
    assessment = None
except APIError as e:
    print(f"Unexpected error: {e.message}")
    raise

Don't Catch What You Can't Handle

# Bad: Swallowing all errors
try:
    assessment = client.create_assessment(...)
except Exception:
    pass  # Lost all error information!

# Good: Only catch specific, handleable errors
try:
    assessment = client.create_assessment(...)
except ValidationFailed as e:
    print(f"Fix parameters: {e.message}")
    raise
except LimitExceeded as e:
    print(f"Upgrade plan: {e.message}")
    raise

Provide User-Friendly Messages

from modelred import ValidationFailed, LimitExceeded, NotAllowedForApiKey

try:
    assessment = client.create_assessment(...)
except ValidationFailed as e:
    return {
        "success": False,
        "message": "Please check your input parameters.",
        "details": e.details,
    }
except LimitExceeded:
    return {
        "success": False,
        "message": "You've reached your plan limit. Please upgrade.",
    }
except NotAllowedForApiKey:
    return {
        "success": False,
        "message": "This action requires web UI access.",
    }

Common Error Scenarios

Invalid API Key

from modelred import Unauthorized

try:
    client = ModelRed(api_key="sk-wrong-prefix")
except ValueError as e:
    print("API key must start with 'mr_'")

try:
    client = ModelRed(api_key="mr_invalid")
    models = client.list_models()
except Unauthorized:
    print("Invalid or expired API key")

Missing Required Parameters

from modelred import ValidationFailed

try:
    assessment = client.create_assessment_by_id(
        model_id="model_123",
        probe_pack_ids=[],  # Empty!
        detector_provider="openai",
        detector_api_key="sk-...",
        detector_model="gpt-4o-mini",
    )
except ValidationFailed as e:
    print("At least one probe pack required")

Resource Not Found

from modelred import NotFound

try:
    pack = client.get_probe_pack("unimported_public_pack")
except NotFound:
    print("Import the probe pack via web UI first")

Plan Limits

from modelred import LimitExceeded

try:
    assessment = client.create_assessment_by_id(...)
except LimitExceeded as e:
    if "assessment" in e.message.lower():
        print("Monthly assessment limit reached")
    elif "model" in e.message.lower():
        print("Model limit reached")

Complete Example

robust_error_handling.py

from modelred import (
    ModelRed,
    ValidationFailed,
    NotFound,
    LimitExceeded,
    RateLimited,
    ServerError,
    APIError,
)
import time
import logging
import os

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

def create_assessment_robust(client, model_id, probe_pack_ids):
    """Create assessment with comprehensive error handling."""
    max_attempts = 3

    for attempt in range(max_attempts):
        try:
            assessment = client.create_assessment_by_id(
                model_id=model_id,
                probe_pack_ids=probe_pack_ids,
                detector_provider="openai",
                detector_api_key=os.environ["OPENAI_API_KEY"],
                detector_model="gpt-4o-mini",
            )

            logger.info(f"Assessment created successfully: {assessment['id']}")
            return assessment

        except ValidationFailed as e:
            logger.error(f"Validation error: {e.message}")
            if e.details:
                logger.error(f"Field errors: {e.details}")
            return None  # Don't retry

        except NotFound as e:
            logger.error(f"Resource not found: {e.message}")
            return None  # Don't retry

        except LimitExceeded as e:
            logger.error(f"Plan limit exceeded: {e.message}")
            return None  # Can't retry without plan upgrade

        except RateLimited as e:
            if attempt < max_attempts - 1:
                wait = 2 ** attempt
                logger.warning(f"Rate limited, waiting {wait}s... (attempt {attempt + 1})")
                time.sleep(wait)
                continue
            logger.error("Max retries exceeded for rate limiting")
            return None

        except ServerError as e:
            if attempt < max_attempts - 1:
                wait = 2 ** attempt
                logger.warning(f"Server error, retrying in {wait}s... (attempt {attempt + 1})")
                time.sleep(wait)
                continue
            logger.error("Max retries exceeded for server errors")
            return None

        except APIError as e:
            logger.error(f"Unexpected API error {e.status}: {e.message}")
            return None

    return None

# Usage
client = ModelRed(api_key=os.environ["MODELRED_API_KEY"])
result = create_assessment_robust(
    client,
    model_id="model_123",
    probe_pack_ids=["pack_1", "pack_2"]
)

if result:
    print(f"✓ Success: {result['id']}")
else:
    print("✗ Failed to create assessment")

Next Steps

Review Best Practices for production deployments
Learn about Pagination for handling large datasets
Explore Python SDK configuration options
Check the FAQ for common troubleshooting tips