Documentation

Model Configuration

Configure and customize your registered AI models

⚙️

Model Configuration & Customization

Fine-tune your model settings with flexible configuration options, provider-specific parameters, and comprehensive metadata management.

ProviderConfig Helper

Provider Configuration Helpers

Use pre-built configuration helpers for consistent, validated provider settings.

🤖

OpenAI Configuration

PYTHON
from modelred import ProviderConfig

config = ProviderConfig.openai(
    api_key="sk-your-key",
    model_name="gpt-4",
    base_url="https://api.openai.com/v1",  # Optional
    organization="org-123"  # Optional
)
☁️

Azure Configuration

PYTHON
config = ProviderConfig.azure(
    api_key="your-azure-key",
    endpoint="https://company.openai.azure.com/",
    deployment_name="gpt-4",
    api_version="2024-02-15-preview"  # Optional
)

Provider-Specific Configurations

🔮 Anthropic Configuration

Basic Setup

PYTHON
config = ProviderConfig.anthropic(
    api_key="sk-ant-your-key",
    model_name="claude-3-sonnet-20240229"
)

Supported Models

claude-3-sonnet-20240229
claude-3-opus-20240229
claude-3-haiku-20240307

🤗 Hugging Face Configuration

Inference API

PYTHON
config = ProviderConfig.huggingface(
    model_name="microsoft/DialoGPT-small",
    api_key="hf_your_token",
    use_inference_api=True,
    task="text-generation"
)

Custom Endpoint

PYTHON
config = ProviderConfig.huggingface(
    model_name="your-model",
    endpoint_url="https://your-endpoint.hf.space",
    use_inference_api=False
)

AWS Service Configuration

AWS Credentials & Regions

Configure AWS services with proper credentials and regional settings.

🏗️

SageMaker

PYTHON
config = ProviderConfig.sagemaker(
    endpoint_name="my-llama-endpoint",
    aws_access_key_id="AKIA...",
    aws_secret_access_key="...",
    region="us-east-1"
)
🏔️

Bedrock

PYTHON
config = ProviderConfig.bedrock(
    model_id="anthropic.claude-v2",
    aws_access_key_id="AKIA...",
    aws_secret_access_key="...",
    region="us-east-1"
)

Custom REST API Configuration

🔗 Custom REST Configuration Options

Configure custom REST endpoints with flexible authentication and request formats.

PYTHON
from modelred import AuthType, RequestFormat

config = ProviderConfig.custom_rest(
endpoint_url="https://api.mycompany.com/v1/chat",
auth_type=AuthType.BEARER,
auth_value="your-bearer-token",
custom_header_name=None, # Optional
headers={"User-Agent": "MyApp/1.0"}, # Optional
request_format=RequestFormat.OPENAI_COMPATIBLE,
response_path="choices.0.message.content",
custom_payload_template={ # Optional
"model": "custom-model",
"messages": "{prompt}",
"max_tokens": 150
}
)

🔐 Authentication Types

Bearer Token

PYTHON
auth_type=AuthType.BEARER,
auth_value="your-bearer-token"

API Key Header

PYTHON
auth_type=AuthType.API_KEY,
auth_value="your-api-key"
# Adds: X-API-Key: your-api-key

Custom Header

PYTHON
auth_type=AuthType.CUSTOM_HEADER,
auth_value="secret-value",
custom_header_name="X-Custom-Auth"

No Authentication

PYTHON
auth_type=AuthType.NONE
# No authentication headers

Request Format Configuration

📋 Request Format Options

OpenAI Compatible

Standard OpenAI chat completions format

RequestFormat.OPENAI_COMPATIBLE

Anthropic Style

Anthropic's message format

RequestFormat.ANTHROPIC_STYLE

Simple Format

Simple prompt-based requests

RequestFormat.SIMPLE

Custom Format

Use custom payload template

RequestFormat.CUSTOM

Environment-Based Configuration

🌍 Environment-Specific Settings

Configure different settings for development, staging, and production environments.

PYTHON
import os
from modelred import ModelRed, ProviderConfig

# Environment-based configuration

ENVIRONMENT = os.getenv("ENVIRONMENT", "development")

async def register_environment_models():
async with ModelRed() as client:

        # Development configuration
        if ENVIRONMENT == "development":
            config = ProviderConfig.openai(
                api_key=os.getenv("OPENAI_API_KEY_DEV"),
                model_name="gpt-3.5-turbo",  # Cheaper for dev
                base_url="https://api.openai.com/v1"
            )
            metadata = {
                "environment": "development",
                "cost_tracking": False,
                "debug_mode": True
            }

        # Production configuration
        elif ENVIRONMENT == "production":
            config = ProviderConfig.openai(
                api_key=os.getenv("OPENAI_API_KEY_PROD"),
                model_name="gpt-4",  # Better model for prod
                organization=os.getenv("OPENAI_ORG_ID")
            )
            metadata = {
                "environment": "production",
                "cost_tracking": True,
                "debug_mode": False,
                "monitoring": "enabled"
            }

        await client.register_model(
            model_id=f"gpt-model-{ENVIRONMENT}",
            provider="OPENAI",
            provider_config=config,
            metadata=metadata
        )

Metadata Configuration

📝 Rich Metadata Structure

Organize models with comprehensive metadata for better management and tracking.

PYTHON
metadata = {
    # Environment & Deployment
    "environment": "production",
    "version": "2.1.0",
    "deployed_at": "2024-01-15T10:30:00Z",
    "region": "us-east-1",

    # Team & Ownership
    "team": "ai-platform",
    "owner": "jane.smith@company.com",
    "project": "customer-assistant",
    "cost_center": "engineering",

    # Technical Configuration
    "max_tokens": 4096,
    "temperature": 0.7,
    "timeout_seconds": 30,
    "retry_attempts": 3,

    # Business Context
    "use_case": "customer_support",
    "criticality": "high",
    "compliance_required": True,
    "data_classification": "sensitive",

    # Features & Capabilities
    "languages": ["en", "es", "fr"],
    "functions_enabled": True,
    "streaming_supported": False,

    # Monitoring & Alerts
    "monitoring_enabled": True,
    "alert_thresholds": {
        "error_rate": 0.05,
        "latency_p99": 2000
    },

    # Custom Tags
    "tags": ["production", "customer-facing", "high-volume"]
}

🏷️ Tagging Strategy

Recommended Tags

Environment (prod, dev, test)
Team/Department ownership
Use case or application
Criticality level

Example Tags

productioncustomer-facinghigh-volumeai-platformcritical

Configuration Validation

Automatic Validation

ModelRed automatically validates configurations to catch common errors early.

Validation Checks

Required fields present

URL format validation

API key format checking

Enum value validation

Error Examples

ValidationError: API key is required

ValidationError: Invalid endpoint URL

ValidationError: Unknown auth type

Configuration Testing

🧪 Test Your Configuration

Validate your configuration before registration with this test pattern.

PYTHON
import asyncio
from modelred import ModelRed, ProviderConfig, ValidationError

async def test_configuration():
"""Test model configuration before registration"""
try: # Test configuration creation
config = ProviderConfig.openai(
api_key="sk-test-key",
model_name="gpt-4",
base_url="https://api.openai.com/v1"
)
print("✅ Configuration created successfully")

        # Test with ModelRed client
        async with ModelRed() as client:
            # This will validate but not actually register
            await client.register_model(
                model_id="test-config-model",
                provider="OPENAI",
                provider_config=config,
                metadata={"test": True}
            )
            print("✅ Configuration validation passed")

    except ValidationError as e:
        print(f"❌ Configuration error: {e}")
    except Exception as e:
        print(f"❌ Unexpected error: {e}")

# Run the test

asyncio.run(test_configuration())

Best Practices

💡 Configuration Best Practices

Security

Use environment variables
Never hardcode API keys
Rotate keys regularly
Use minimum required permissions

Organization

Use descriptive metadata
Tag by environment and team
Document business context
Include ownership information

Next Steps