AWS Unveils New AI Tools at re:Invent Conference
AWS has announced a slew of new updates to its AI tools during its re:Invent conference, including enhancements to its SageMaker HyperPod AI model training environment, as well as to Bedrock, its environment for building generative AI applications using foundation models.
SageMaker HyperPod Updates
AWS has brought many GenAI capabilities to its cloud, and the rollout continued this week. The company unveiled several enhancements to SageMaker HyperPod, which it first launched a year ago to speed the training of foundation models. Different AI teams have different training needs. Some teams may need a large amount of accelerated compute for a short amount of time, while others may need smaller amounts over a longer period of time. With the new task governance capability, AI development teams can create flexible training plans that SageMaker HyperPod will then execute using EC2 capacity blocks.
The new capability will dynamically allocate workload to enable customers to get more useful work out of their large clusters at certain times, such as when data scientists and AI engineers go to sleep. "Normally you don’t want these expensive systems sitting idle," said Rahul Pathak, VP of data and AI at AWS. "The new task governance capability will be a game-changer for our customers, allowing them to optimize their compute resources and reduce costs."
Bedrock Updates
AWS also made various announcements for Bedrock, the collection of tools it launched in April 2023 for building generative AI applications using its own pre-trained foundation models, such as Titan, as well as third-party models from AI21 Labs, Anthropic, and Stability AI, among others.
Bedrock customers can use the new Nova family of models that AWS announced on Tuesday, including Nova Micro, Nova Lite, Nova Pro, Nova Premier, Nova Canvas, and Amazon Nova Reel. Customers can also use foundation models from Poolside, Stability AI, and Luma AI, and dozens more via Bedrock Marketplace, which AWS also launched today. AWS says Bedrock Marketplace currently has more than 100 models.
New Features in Bedrock
To help save customers money when submitting the same prompt over and over, AWS unveiled a new Bedrock feature called prompt caching. According to Pathak, by automatically caching repetitive prompts, AWS can not only reduce costs by up to 90% for Bedrock users, but it can reduce latency by up to 85%.
AI models can be unpredictable; that’s the nature of probabilistic systems. To prevent some of the worst behaviors, AWS has supported guardrails on Bedrock, but only for language models. Today, it updated the guardrails to support multi-modal toxicity detection in images generated with Bedrock foundation models.
Bedrock Data Automation (BDA) is another capability unveiled today that allows Bedrock Knowledge Base to support unstructured data, such as documents, images, and data held in tables, into their GenAI apps. The new Bedrock feature should make it easier for developers to build intelligent document processing, media analysis, and other multimodal data-centric automation solutions, AWS said.
Conclusion
The new updates to SageMaker HyperPod and Bedrock aim to make it easier for AI teams to build and train AI models, and for developers to build generative AI applications using foundation models. With the new task governance capability, Bedrock users can now optimize their compute resources and reduce costs. The new features in Bedrock, such as prompt caching, guardrails, and BDA, will help reduce costs and improve latency. The updates are designed to make it easier for developers to build intelligent applications that can process and analyze unstructured data.
FAQs
Q: What is the new task governance capability in SageMaker HyperPod?
A: The new task governance capability in SageMaker HyperPod allows AI development teams to create flexible training plans that can be executed using EC2 capacity blocks, allowing customers to dynamically allocate workload and optimize their compute resources.
Q: What is Bedrock Data Automation (BDA)?
A: BDA is a new capability in Bedrock that allows Bedrock Knowledge Base to support unstructured data, such as documents, images, and data held in tables, into their GenAI apps.
Q: How does prompt caching in Bedrock work?
A: Bedrock’s prompt caching feature automatically caches repetitive prompts, reducing costs by up to 90% and latency by up to 85%.
Q: What is the purpose of the new guardrails in Bedrock?
A: The new guardrails in Bedrock are designed to prevent some of the worst behaviors of AI models, such as multi-modal toxicity detection in images generated with Bedrock foundation models.
