Documentation

Common Issues

Frequently encountered problems and their solutions

📋

Frequently Asked Questions

Quick solutions to the most common ModelRed SDK issues with copy-paste fixes and step-by-step guidance.

Installation Problems

"ModuleNotFoundError: No module named 'modelred'"

Problem: SDK not installed or wrong Python environment

SOLUTION 1: Install SDK
pip install modelred
SOLUTION 2: Check Virtual Environment
which python
pip list | grep modelred
SOLUTION 3: Force Reinstall
pip uninstall modelred && pip install modelred

"ImportError: cannot import name 'ModelRed'"

Problem: Incorrect import syntax or outdated version

CORRECT IMPORT
# ✅ Correct
from modelred import ModelRed

# ❌ Wrong
import ModelRed
from ModelRed import ModelRed

Configuration Issues

🔧 "API key required" or "Invalid API key format"

Problem: Missing or incorrectly formatted API key

SOLUTION 1: Environment Variable
# Set in terminal
export MODELRED_API_KEY="mr_your_api_key_here"

# Or add to .env file
echo "MODELRED_API_KEY=mr_your_api_key_here" >> .env
SOLUTION 2: Direct Parameter
# Pass directly to client
async with ModelRed(api_key="mr_your_api_key_here") as client:
    # Your code here
SOLUTION 3: Verify Format

✅ Valid format: mr_1234567890abcdef...

❌ Invalid: Missing "mr_" prefix

❌ Invalid: Too short (less than 32 chars)

🔧 ".env file not loading"

Problem: Environment variables not accessible

SOLUTION: Load .env File
from dotenv import load_dotenv
import os

# Load .env file
load_dotenv()

# Now environment variables are available
api_key = os.getenv("MODELRED_API_KEY")
print(f"API Key loaded: {api_key is not None}")

Authentication Failures

🔑 "AuthenticationError: Invalid API key"

Problem: API key validation failed

DIAGNOSTIC SCRIPT
import os
from modelred import ModelRed

async def diagnose_auth():
    # Check environment
    api_key = os.getenv("MODELRED_API_KEY")
    print(f"API key found: {api_key is not None}")

    if api_key:
        print(f"Key length: {len(api_key)}")
        print(f"Starts with 'mr_': {api_key.startswith('mr_')}")

        # Test authentication
        try:
            async with ModelRed() as client:
                account = await client.validate_api_key()
                print(f"✅ Authentication successful")
                print(f"Organization: {account.get('organization')}")
        except Exception as e:
            print(f"❌ Authentication failed: {e}")
            print("💡 Get a new API key from ModelRed dashboard")
    else:
        print("❌ No API key found in environment")

import asyncio
asyncio.run(diagnose_auth())

🔑 "TierRestrictedError: Feature requires higher tier"

Problem: Subscription tier insufficient for requested feature

CHECK TIER REQUIREMENTS
async def check_tier_requirements():
    async with ModelRed() as client:
        # Check current tier
        usage = await client.get_usage_stats()
        print(f"Current tier: {usage.tier}")

        # Check provider requirements
        providers = await client.get_providers()
        for provider in providers:
            print(f"{provider.display_name}: {provider.tier_required} tier required")

Model & Provider Issues

🤖 "ModelNotFoundError: Model doesn't exist"

Problem: Model ID not registered or incorrect

FIND CORRECT MODEL ID
async def find_models():
    async with ModelRed() as client:
        models = await client.list_models()

        if not models:
            print("❌ No models registered")
            print("💡 Register a model first")
        else:
            print("📋 Available models:")
            for model in models:
                print(f"   ID: {model['modelId']}")
                print(f"   Name: {model.get('displayName', 'Unknown')}")
                print(f"   Provider: {model['provider']}")
                print()

🤖 Provider API key errors (OpenAI, Anthropic, etc.)

Problem: Third-party provider authentication failing

OPENAI ISSUES

• Ensure API key starts with "sk-"

• Check OpenAI account has credits

• Verify model name is correct (e.g., "gpt-3.5-turbo")

ANTHROPIC ISSUES

• Ensure API key starts with "sk-ant-"

• Check model name format (e.g., "claude-3-sonnet-20240229")

• Verify Anthropic account status

AZURE OPENAI ISSUES

• Check endpoint URL format

• Verify deployment name matches Azure setup

• Ensure API version is supported

Assessment Problems

🧪 "AssessmentError: Assessment failed to start"

Problem: Assessment won't initialize

PRE-FLIGHT CHECK
async def preflight_check(model_id, test_suites):
    async with ModelRed() as client:
        print("🔍 Pre-flight Assessment Check")

        # 1. Check model exists
        try:
            model = await client.get_model(model_id)
            print(f"✅ Model found: {model['displayName']}")
        except:
            print(f"❌ Model '{model_id}' not found")
            return False

        # 2. Check test suites
        available = await client.get_test_suites()
        available_names = [s['name'] for s in available]

        for suite in test_suites:
            if suite in available_names:
                print(f"✅ Test suite available: {suite}")
            else:
                print(f"❌ Test suite unavailable: {suite}")
                print(f"   Available: {available_names}")
                return False

        # 3. Check quota
        usage = await client.get_usage_stats()
        if usage.assessments_this_month >= usage.assessments_limit:
            print("❌ Assessment quota exceeded")
            return False

        print("✅ Pre-flight check passed")
        return True

🧪 Assessment gets stuck at 0% or doesn't progress

Problem: Assessment starts but makes no progress

COMMON CAUSES

• Provider API key expired or invalid

• Model endpoint not responding

• Network connectivity issues

• Provider rate limiting

QUICK FIXES

1. Cancel and restart assessment

2. Try with basic_security test suite only

3. Check provider dashboard for issues

4. Test model connectivity manually

Network & Connection Issues

🌐 "Connection timeout" or "Network error"

Problem: Cannot reach ModelRed API or provider APIs

CONNECTIVITY TEST
import aiohttp
import asyncio

async def test_connections():
    """Test connectivity to key endpoints"""
    endpoints = [
        ("ModelRed API", "http://localhost:3000/api/health"),
        ("OpenAI", "https://api.openai.com/v1/models"),
        ("Anthropic", "https://api.anthropic.com/v1/models")
    ]

    timeout = aiohttp.ClientTimeout(total=10)

    for name, url in endpoints:
        try:
            async with aiohttp.ClientSession(timeout=timeout) as session:
                async with session.get(url) as response:
                    print(f"✅ {name}: Reachable (HTTP {response.status})")
        except asyncio.TimeoutError:
            print(f"❌ {name}: Timeout (check firewall/proxy)")
        except Exception as e:
            print(f"❌ {name}: Error ({type(e).__name__})")

await test_connections()

Performance Issues

Slow response times or timeouts

Problem: Operations taking too long or timing out

QUICK FIXES

1. Increase timeout values in your code

2. Use smaller test suites for assessments

3. Check internet connection speed

4. Restart Python interpreter to clear cache

TIMEOUT ADJUSTMENT
# Increase timeout for slow connections
result = await client.run_assessment(
    model_id="your-model",
    test_suites=["basic_security"],
    timeout_minutes=60,  # Increase from default 30
    wait_for_completion=True
)

Quick Reference Solutions

Quick Fixes Checklist

First Things to Try

1.Restart Python interpreter
2.Check API key format
3.Verify internet connection
4.Update ModelRed SDK

Emergency Commands

pip install --upgrade modelred
pip uninstall modelred && pip install modelred
echo $MODELRED_API_KEY

Getting Help

🆘 Still Need Help?

Before Contacting Support

✅ Run the health check script from troubleshooting

✅ Try the quick fixes above

✅ Check this common issues list

✅ Note the exact error message

Include in Support Request

• Operating system and Python version

• ModelRed SDK version

• Complete error message and stack trace

• Code snippet that reproduces the issue

• What you were trying to accomplish

Next Steps