Model Configuration
Configure and customize your registered AI models
Model Configuration & Customization
Fine-tune your model settings with flexible configuration options, provider-specific parameters, and comprehensive metadata management.
ProviderConfig Helper
Provider Configuration Helpers
Use pre-built configuration helpers for consistent, validated provider settings.
OpenAI Configuration
from modelred import ProviderConfig
config = ProviderConfig.openai(
api_key="sk-your-key",
model_name="gpt-4",
base_url="https://api.openai.com/v1", # Optional
organization="org-123" # Optional
)
Azure Configuration
config = ProviderConfig.azure(
api_key="your-azure-key",
endpoint="https://company.openai.azure.com/",
deployment_name="gpt-4",
api_version="2024-02-15-preview" # Optional
)
Provider-Specific Configurations
🔮 Anthropic Configuration
Basic Setup
config = ProviderConfig.anthropic(
api_key="sk-ant-your-key",
model_name="claude-3-sonnet-20240229"
)
Supported Models
claude-3-sonnet-20240229
claude-3-opus-20240229
claude-3-haiku-20240307
🤗 Hugging Face Configuration
Inference API
config = ProviderConfig.huggingface(
model_name="microsoft/DialoGPT-small",
api_key="hf_your_token",
use_inference_api=True,
task="text-generation"
)
Custom Endpoint
config = ProviderConfig.huggingface(
model_name="your-model",
endpoint_url="https://your-endpoint.hf.space",
use_inference_api=False
)
AWS Service Configuration
AWS Credentials & Regions
Configure AWS services with proper credentials and regional settings.
SageMaker
config = ProviderConfig.sagemaker(
endpoint_name="my-llama-endpoint",
aws_access_key_id="AKIA...",
aws_secret_access_key="...",
region="us-east-1"
)
Bedrock
config = ProviderConfig.bedrock(
model_id="anthropic.claude-v2",
aws_access_key_id="AKIA...",
aws_secret_access_key="...",
region="us-east-1"
)
Custom REST API Configuration
🔗 Custom REST Configuration Options
Configure custom REST endpoints with flexible authentication and request formats.
from modelred import AuthType, RequestFormat
config = ProviderConfig.custom_rest(
endpoint_url="https://api.mycompany.com/v1/chat",
auth_type=AuthType.BEARER,
auth_value="your-bearer-token",
custom_header_name=None, # Optional
headers={"User-Agent": "MyApp/1.0"}, # Optional
request_format=RequestFormat.OPENAI_COMPATIBLE,
response_path="choices.0.message.content",
custom_payload_template={ # Optional
"model": "custom-model",
"messages": "{prompt}",
"max_tokens": 150
}
)
🔐 Authentication Types
Bearer Token
auth_type=AuthType.BEARER,
auth_value="your-bearer-token"
API Key Header
auth_type=AuthType.API_KEY,
auth_value="your-api-key"
# Adds: X-API-Key: your-api-key
Custom Header
auth_type=AuthType.CUSTOM_HEADER,
auth_value="secret-value",
custom_header_name="X-Custom-Auth"
No Authentication
auth_type=AuthType.NONE
# No authentication headers
Request Format Configuration
📋 Request Format Options
OpenAI Compatible
Standard OpenAI chat completions format
RequestFormat.OPENAI_COMPATIBLE
Anthropic Style
Anthropic's message format
RequestFormat.ANTHROPIC_STYLE
Simple Format
Simple prompt-based requests
RequestFormat.SIMPLE
Custom Format
Use custom payload template
RequestFormat.CUSTOM
Environment-Based Configuration
🌍 Environment-Specific Settings
Configure different settings for development, staging, and production environments.
import os
from modelred import ModelRed, ProviderConfig
# Environment-based configuration
ENVIRONMENT = os.getenv("ENVIRONMENT", "development")
async def register_environment_models():
async with ModelRed() as client:
# Development configuration
if ENVIRONMENT == "development":
config = ProviderConfig.openai(
api_key=os.getenv("OPENAI_API_KEY_DEV"),
model_name="gpt-3.5-turbo", # Cheaper for dev
base_url="https://api.openai.com/v1"
)
metadata = {
"environment": "development",
"cost_tracking": False,
"debug_mode": True
}
# Production configuration
elif ENVIRONMENT == "production":
config = ProviderConfig.openai(
api_key=os.getenv("OPENAI_API_KEY_PROD"),
model_name="gpt-4", # Better model for prod
organization=os.getenv("OPENAI_ORG_ID")
)
metadata = {
"environment": "production",
"cost_tracking": True,
"debug_mode": False,
"monitoring": "enabled"
}
await client.register_model(
model_id=f"gpt-model-{ENVIRONMENT}",
provider="OPENAI",
provider_config=config,
metadata=metadata
)
Metadata Configuration
📝 Rich Metadata Structure
Organize models with comprehensive metadata for better management and tracking.
metadata = {
# Environment & Deployment
"environment": "production",
"version": "2.1.0",
"deployed_at": "2024-01-15T10:30:00Z",
"region": "us-east-1",
# Team & Ownership
"team": "ai-platform",
"owner": "jane.smith@company.com",
"project": "customer-assistant",
"cost_center": "engineering",
# Technical Configuration
"max_tokens": 4096,
"temperature": 0.7,
"timeout_seconds": 30,
"retry_attempts": 3,
# Business Context
"use_case": "customer_support",
"criticality": "high",
"compliance_required": True,
"data_classification": "sensitive",
# Features & Capabilities
"languages": ["en", "es", "fr"],
"functions_enabled": True,
"streaming_supported": False,
# Monitoring & Alerts
"monitoring_enabled": True,
"alert_thresholds": {
"error_rate": 0.05,
"latency_p99": 2000
},
# Custom Tags
"tags": ["production", "customer-facing", "high-volume"]
}
🏷️ Tagging Strategy
Recommended Tags
Example Tags
Configuration Validation
✅ Automatic Validation
ModelRed automatically validates configurations to catch common errors early.
Validation Checks
Required fields present
URL format validation
API key format checking
Enum value validation
Error Examples
ValidationError: API key is required
ValidationError: Invalid endpoint URL
ValidationError: Unknown auth type
Configuration Testing
🧪 Test Your Configuration
Validate your configuration before registration with this test pattern.
import asyncio
from modelred import ModelRed, ProviderConfig, ValidationError
async def test_configuration():
"""Test model configuration before registration"""
try: # Test configuration creation
config = ProviderConfig.openai(
api_key="sk-test-key",
model_name="gpt-4",
base_url="https://api.openai.com/v1"
)
print("✅ Configuration created successfully")
# Test with ModelRed client
async with ModelRed() as client:
# This will validate but not actually register
await client.register_model(
model_id="test-config-model",
provider="OPENAI",
provider_config=config,
metadata={"test": True}
)
print("✅ Configuration validation passed")
except ValidationError as e:
print(f"❌ Configuration error: {e}")
except Exception as e:
print(f"❌ Unexpected error: {e}")
# Run the test
asyncio.run(test_configuration())
Best Practices
💡 Configuration Best Practices
Security
Organization
Next Steps
Model Management
Learn to list and manage your models
Start Testing
Run security assessments on your models