Streamlining AI Model Deployment Workflows with Amazon Bedrock Custom Model Import
In enterprise environments, organizations often divide their AI operations into two specialized teams: an AI research team and a model hosting team. The research team is dedicated to developing and enhancing AI models using model training and fine-tuning techniques. Meanwhile, a separate hosting team is responsible for deploying these models across their own development, staging, and production environments.
Amazon Bedrock Custom Model Import
With Amazon Bedrock Custom Model Import, the hosting team can import and serve custom models using supported architectures such as Meta Llama 2, Llama 3, and Mistral using On-Demand pricing. Teams can import models with weights in Hugging Face safetensors format from Amazon SageMaker or from Amazon Simple Storage Service (Amazon S3). These imported custom models work alongside existing Amazon Bedrock foundation models (FMs) through a single, unified API in a serverless manner, alleviating the need to manage model deployment and scaling.
Cross-Account Access
In such enterprise environments, these teams often work in separate AWS accounts for security and operational reasons. The model development team’s training results, known as model artifacts, for example model weights, are typically stored in S3 buckets within the research team’s AWS account, but the hosting team needs to access these artifacts from another account to deploy models. This creates a challenge: how do you securely share model artifacts between accounts?
Solving the Challenge
Amazon Bedrock Custom Model Import cross-account support helps you configure direct access between the S3 buckets storing model artifacts and the hosting account. This streamlines your operational workflow while maintaining security boundaries between teams. One of our customers quotes:
"Bedrock Custom Model Import cross-account support helped AI Platform team to simplify the configuration, reduce operational overhead and secure models in the original location."
– Scott Chang, Principal Engineer, AI Platform at Salesforce
Prerequisites
Before starting a custom model import job, you need to fulfill the following prerequisites:
- If you’re importing your model from an S3 bucket, prepare your model files in the Hugging Face weights format. For more information, refer to Import source.
- (Optional) Set up extra security configurations.
Step-by-Step Execution
The following section provides the step-by-step execution of the previously outlined high-level process, from the perspective of an administrator managing both accounts:
Step 1: Set up the S3 bucket policy (in the Model Development account) to enable access for the Model Hosting account’s IAM role:
- Sign in to the AWS Management Console for account 111122223333, then access the Amazon S3 console.
- On the General purpose buckets view, locate model-artifacts-111122223333, the bucket used by the model development team to store their model artifacts.
- On the Permissions tab, select Edit in the Bucket policy section, and insert the following IAM resource-based policy…
Cleanup
There is no additional charge to import a custom model to Amazon Bedrock (refer to step 6 in the Step-by-Step Execution section). However, if your model isn’t in use for inference, and you want to avoid paying storage costs (refer to Amazon Bedrock pricing), delete the imported model using the AWS console or AWS CLI reference or API Reference. For example (replace the text in red with your imported model name):
aws bedrock delete-imported-model \
–model-identifier "mistral-777788889999-01"
Conclusion
By using cross-account access in Amazon Bedrock Custom Model Import, organizations can significantly streamline their AI model deployment workflows.
About the Authors
Hrushikesh Gangur is a Principal Solutions Architect at AWS. He holds a master’s degree in Computer Engineering from Cornell University, where he worked in the Autonomous Systems Lab with a specialization in computer vision and robot perception. Currently, he helps deploy large language models to optimize throughput and latency.
Sai Darahas Akkineni is a Software Development Engineer at AWS. He holds a master’s degree in Computer Engineering from Cornell University, where he worked in the Autonomous Systems Lab with a specialization in computer vision and robot perception. Currently, he helps deploy large language models to optimize throughput and latency.
Prashant Patel is a Senior Software Development Engineer in AWS. He’s passionate about scaling large language models for enterprise applications. Prior to joining AWS, he worked at IBM on productionizing large-scale AI/ML workloads on Kubernetes. Prashant has a master’s degree from NYU Tandon School of Engineering. While not at work, he enjoys traveling and playing with his dogs.
FAQs
Q: What is Amazon Bedrock Custom Model Import?
A: Amazon Bedrock Custom Model Import allows you to import and serve custom models using supported architectures such as Meta Llama 2, Llama 3, and Mistral using On-Demand pricing.
Q: How do I securely share model artifacts between accounts?
A: Amazon Bedrock Custom Model Import cross-account support helps you configure direct access between the S3 buckets storing model artifacts and the hosting account.
Q: What are the prerequisites for starting a custom model import job?
A: The prerequisites include preparing your model files in the Hugging Face weights format and (optional) setting up extra security configurations.
