Documentation

AWS SageMaker

Integrate AWS SageMaker endpoints with ModelRed

🏗️

AWS SageMaker Endpoints

Test custom models deployed on SageMaker with enterprise-grade infrastructure, auto-scaling, and built-in monitoring capabilities.

Quick Setup

Get Started in 4 Steps

Connect your SageMaker endpoint for security testing.

1

Deploy Your Model

Deploy your model to a SageMaker endpoint through the AWS Console or CLI.

Endpoint Name: my-model-endpoint
2

Get AWS Credentials

Set up AWS IAM credentials with SageMaker access permissions.

Access Key ID: AKIA...Secret Access Key: ...Region: us-east-1
3

Set Environment Variables

BASH
export AWS_ACCESS_KEY_ID="AKIA..."
export AWS_SECRET_ACCESS_KEY="your-secret-key"
export AWS_DEFAULT_REGION="us-east-1"
4

Register Your Endpoint

PYTHON
from modelred import ModelRed

async with ModelRed() as client:
    await client.register_sagemaker_model(
        model_id="my-sagemaker-model",
        endpoint_name="my-model-endpoint",
        aws_access_key_id="AKIA...",  # or use env vars
        aws_secret_access_key="your-secret-key",
        region="us-east-1"
    )

Common Model Types

🦙

LLaMA Models

Self-hosted LLaMA variants on SageMaker

PopularCustom
🧠

Fine-tuned Models

Custom fine-tuned language models

EnterpriseCustom
⚡

Proprietary Models

Company-specific AI models

PrivateSecure

Configuration Options

Advanced Setup

Different ways to configure your SageMaker endpoints.

🔧

Multiple Endpoints

PYTHON
# Register multiple SageMaker endpoints
endpoints = [
    ("dev-llama", "llama-dev-endpoint"),
    ("prod-llama", "llama-prod-endpoint"),
    ("custom-model", "custom-endpoint-name")
]

for model_id, endpoint_name in endpoints:
    await client.register_sagemaker_model(
        model_id=model_id,
        endpoint_name=endpoint_name,
        region="us-east-1"
    )
🌍

Multi-Region Setup

PYTHON
# Different regions for compliance
await client.register_sagemaker_model(
    model_id="eu-model",
    endpoint_name="eu-compliance-model",
    region="eu-west-1",
    metadata={
        "compliance": "GDPR",
        "data_residency": "EU"
    }
)

Enterprise Features

🏢 SageMaker Advantages

Infrastructure & Scaling

•

Auto-scaling capabilities

•

Multi-AZ deployments

•Built-in monitoring
•A/B testing support

Security & Compliance

•VPC isolation
•IAM access control
•

Encryption at rest/transit

•

Compliance certifications

Common Issues

⚠️ Troubleshooting

Endpoint Not Found
EndpointNotFound: Could not find endpoint
Solutions:
  • • Verify endpoint name in SageMaker Console
  • • Check endpoint status is 'InService'
  • • Ensure correct AWS region
Access Denied
AccessDenied: User not authorized
Solutions:
  • • Add SageMaker:InvokeEndpoint permission
  • • Verify AWS credentials are correct
  • • Check IAM policy allows endpoint access
Model Error
ModelError: Endpoint returned 5xx error
Solutions:
  • • Check CloudWatch logs for endpoint errors
  • • Verify model accepts text input format
  • • Ensure sufficient endpoint capacity

Quick Test

✅ Verify Your Setup

Run this test to confirm your SageMaker integration is working:

PYTHON
import asyncio
from modelred import ModelRed

async def test_sagemaker():
    async with ModelRed() as client:
        # Register SageMaker endpoint
        await client.register_sagemaker_model(
            model_id="test-sagemaker",
            endpoint_name="your-endpoint-name",
            region="us-east-1"
        )
        print("✅ SageMaker endpoint registered!")

        # Run security test
        result = await client.run_assessment(
            model_id="test-sagemaker",
            test_suites=["basic_security"]
        )
        print(f"🔍 Assessment started: {result.assessment_id}")

asyncio.run(test_sagemaker())

Next Steps

🧪

Run Security Tests

Start testing your SageMaker endpoints

🏔️

Try AWS Bedrock

Explore managed foundation models